[==========] Running 14 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 14 tests from AutoRebalancerTest
[ RUN ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251028 21:09:52.949589 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:39915,127.15.205.61:40097,127.15.205.60:38153
I20251028 21:09:52.949890 16180 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 21:09:52.950021 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:52.953954 16185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:52.953998 16188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:52.954082 16180 server_base.cc:1047] running on GCE node
W20251028 21:09:52.954025 16186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:52.954324 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:52.954365 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:52.954382 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685792954383 us; error 0 us; skew 500 ppm
I20251028 21:09:52.955348 16180 webserver.cc:492] Webserver started at http://127.15.205.62:35881/ using document root <none> and password file <none>
I20251028 21:09:52.955456 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:52.955489 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:52.955543 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:52.956389 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "580543d2bfd24c358e5dba831bd98f07"
format_stamp: "Formatted at 2025-10-28 21:09:52 on dist-test-slave-zmkx"
I20251028 21:09:52.957520 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20251028 21:09:52.957985 16193 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:52.958141 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:52.958220 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "580543d2bfd24c358e5dba831bd98f07"
format_stamp: "Formatted at 2025-10-28 21:09:52 on dist-test-slave-zmkx"
I20251028 21:09:52.958272 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:52.970973 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:52.971222 16180 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 21:09:52.971350 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:52.974953 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:39915
I20251028 21:09:52.974988 16245 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:39915 every 8 connection(s)
I20251028 21:09:52.975368 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:09:52.975556 16246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 21:09:52.976397 16251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:52.976406 16248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:52.976634 16180 server_base.cc:1047] running on GCE node
W20251028 21:09:52.976531 16249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:52.976830 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:52.976867 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:52.976888 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685792976889 us; error 0 us; skew 500 ppm
I20251028 21:09:52.977434 16180 webserver.cc:492] Webserver started at http://127.15.205.61:34921/ using document root <none> and password file <none>
I20251028 21:09:52.977516 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:52.977557 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:52.977599 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:52.977802 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root/instance:
uuid: "1122ffcba2914659812e449048dd8486"
format_stamp: "Formatted at 2025-10-28 21:09:52 on dist-test-slave-zmkx"
I20251028 21:09:52.977929 16246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:09:52.978619 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:52.979106 16258 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:52.979319 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:52.979382 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root
uuid: "1122ffcba2914659812e449048dd8486"
format_stamp: "Formatted at 2025-10-28 21:09:52 on dist-test-slave-zmkx"
I20251028 21:09:52.979434 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:52.986411 16246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } has no permanent_uuid. Determining permanent_uuid...
W20251028 21:09:52.986712 16195 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.205.61:40097: connect: Connection refused (error 111)
W20251028 21:09:52.987039 16246 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.205.61:40097: Network error: Client connection negotiation failed: client connection to 127.15.205.61:40097: connect: Connection refused (error 111)
I20251028 21:09:52.998566 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:52.998775 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:53.001873 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.61:40097
I20251028 21:09:53.001951 16311 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.61:40097 every 8 connection(s)
I20251028 21:09:53.002182 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:09:53.002157 16312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:53.002938 16312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } has no permanent_uuid. Determining permanent_uuid...
W20251028 21:09:53.003140 16314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:53.004015 16317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:53.004097 16315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:53.004132 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:53.004315 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:53.004361 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:53.004381 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685793004381 us; error 0 us; skew 500 ppm
I20251028 21:09:53.004848 16180 webserver.cc:492] Webserver started at http://127.15.205.60:41345/ using document root <none> and password file <none>
I20251028 21:09:53.004923 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:53.004967 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:53.005015 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:53.005277 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root/instance:
uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2"
format_stamp: "Formatted at 2025-10-28 21:09:53 on dist-test-slave-zmkx"
I20251028 21:09:53.005990 16312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:09:53.006318 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:53.006886 16324 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.007027 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.007073 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root
uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2"
format_stamp: "Formatted at 2025-10-28 21:09:53 on dist-test-slave-zmkx"
I20251028 21:09:53.007133 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:53.009377 16312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
W20251028 21:09:53.009879 16312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.205.60:38153: Network error: Client connection negotiation failed: client connection to 127.15.205.60:38153: connect: Connection refused (error 111)
I20251028 21:09:53.023871 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:53.024071 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:53.027357 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.60:38153
I20251028 21:09:53.027415 16377 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.60:38153 every 8 connection(s)
I20251028 21:09:53.027558 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:09:53.027643 16378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:53.028182 16378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:09:53.030742 16378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:09:53.033066 16378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:09:53.036057 16378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: Bootstrap starting.
I20251028 21:09:53.036185 16246 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } attempt: 1
I20251028 21:09:53.036938 16378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:53.037175 16378 log.cc:826] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: Log is configured to *not* fsync() on all Append() calls
I20251028 21:09:53.037773 16378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: No bootstrap required, opened a new log
I20251028 21:09:53.038758 16246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:09:53.039309 16378 raft_consensus.cc:359] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.039415 16378 raft_consensus.cc:385] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:53.039451 16378 raft_consensus.cc:740] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f0488d9b3c4c97b95c970d0e0bf8d2, State: Initialized, Role: FOLLOWER
I20251028 21:09:53.039535 16378 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.039734 16383 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.039778 16378 sys_catalog.cc:565] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:09:53.039820 16383 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [sys.catalog]: This master's current role is: FOLLOWER
W20251028 21:09:53.041611 16394 catalog_manager.cc:1568] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 21:09:53.041666 16394 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 21:09:53.041853 16246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07: Bootstrap starting.
I20251028 21:09:53.042230 16246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:53.042768 16246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07: No bootstrap required, opened a new log
I20251028 21:09:53.042910 16246 raft_consensus.cc:359] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.042956 16246 raft_consensus.cc:385] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:53.042979 16246 raft_consensus.cc:740] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580543d2bfd24c358e5dba831bd98f07, State: Initialized, Role: FOLLOWER
I20251028 21:09:53.043030 16246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.043200 16246 sys_catalog.cc:565] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:09:53.043216 16397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.043311 16397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:09:53.044342 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251028 21:09:53.044473 16408 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 21:09:53.044524 16408 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 21:09:53.062036 16312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } attempt: 1
I20251028 21:09:53.065244 16312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486: Bootstrap starting.
I20251028 21:09:53.065785 16312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:53.066387 16312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486: No bootstrap required, opened a new log
I20251028 21:09:53.066536 16312 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.066596 16312 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:53.066617 16312 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1122ffcba2914659812e449048dd8486, State: Initialized, Role: FOLLOWER
I20251028 21:09:53.066665 16312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.066849 16312 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:09:53.066833 16410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.066910 16410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:09:53.068218 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20251028 21:09:53.068307 16421 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 21:09:53.068346 16421 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 21:09:53.068495 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:53.069633 16423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:53.069579 16422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:53.069797 16425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:53.069988 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:53.070094 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:53.070122 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:53.070142 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685793070142 us; error 0 us; skew 500 ppm
I20251028 21:09:53.070719 16180 webserver.cc:492] Webserver started at http://127.15.205.1:46419/ using document root <none> and password file <none>
I20251028 21:09:53.070797 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:53.070838 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:53.070901 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:53.071122 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "2d19753cb4694ba6bb043e03d1a0e926"
format_stamp: "Formatted at 2025-10-28 21:09:53 on dist-test-slave-zmkx"
I20251028 21:09:53.071965 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:53.072378 16430 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.072522 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.072575 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "2d19753cb4694ba6bb043e03d1a0e926"
format_stamp: "Formatted at 2025-10-28 21:09:53 on dist-test-slave-zmkx"
I20251028 21:09:53.072619 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:53.072810 16410 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:53.072892 16410 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.073118 16410 leader_election.cc:290] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580543d2bfd24c358e5dba831bd98f07 (127.15.205.62:39915), d2f0488d9b3c4c97b95c970d0e0bf8d2 (127.15.205.60:38153)
I20251028 21:09:53.073403 16220 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1122ffcba2914659812e449048dd8486" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580543d2bfd24c358e5dba831bd98f07" is_pre_election: true
I20251028 21:09:53.073493 16352 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1122ffcba2914659812e449048dd8486" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" is_pre_election: true
I20251028 21:09:53.073545 16352 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1122ffcba2914659812e449048dd8486 in term 0.
I20251028 21:09:53.073679 16261 leader_election.cc:304] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1122ffcba2914659812e449048dd8486, d2f0488d9b3c4c97b95c970d0e0bf8d2; no voters:
I20251028 21:09:53.073498 16220 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1122ffcba2914659812e449048dd8486 in term 0.
I20251028 21:09:53.073778 16410 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:53.073817 16410 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:53.073844 16410 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:53.074357 16410 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.074447 16410 leader_election.cc:290] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [CANDIDATE]: Term 1 election: Requested vote from peers 580543d2bfd24c358e5dba831bd98f07 (127.15.205.62:39915), d2f0488d9b3c4c97b95c970d0e0bf8d2 (127.15.205.60:38153)
I20251028 21:09:53.074589 16220 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1122ffcba2914659812e449048dd8486" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580543d2bfd24c358e5dba831bd98f07"
I20251028 21:09:53.074648 16220 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:53.074707 16352 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1122ffcba2914659812e449048dd8486" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2"
I20251028 21:09:53.074753 16352 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:53.075314 16220 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1122ffcba2914659812e449048dd8486 in term 1.
I20251028 21:09:53.075388 16352 raft_consensus.cc:2468] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1122ffcba2914659812e449048dd8486 in term 1.
I20251028 21:09:53.075506 16261 leader_election.cc:304] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1122ffcba2914659812e449048dd8486, d2f0488d9b3c4c97b95c970d0e0bf8d2; no voters:
I20251028 21:09:53.075623 16410 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:53.075744 16410 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 1 LEADER]: Becoming Leader. State: Replica: 1122ffcba2914659812e449048dd8486, State: Running, Role: LEADER
I20251028 21:09:53.075812 16410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } }
I20251028 21:09:53.076186 16435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1122ffcba2914659812e449048dd8486. Latest consensus state: current_term: 1 leader_uuid: "1122ffcba2914659812e449048dd8486" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.076259 16435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: This master's current role is: LEADER
I20251028 21:09:53.076427 16437 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:09:53.076630 16437 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:09:53.077630 16352 raft_consensus.cc:1275] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 1 FOLLOWER]: Refusing update from remote peer 1122ffcba2914659812e449048dd8486: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:09:53.077653 16220 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 1 FOLLOWER]: Refusing update from remote peer 1122ffcba2914659812e449048dd8486: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:09:53.077812 16435 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:53.077890 16410 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:53.078383 16397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1122ffcba2914659812e449048dd8486. Latest consensus state: current_term: 1 leader_uuid: "1122ffcba2914659812e449048dd8486" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.078464 16397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:09:53.078383 16383 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1122ffcba2914659812e449048dd8486. Latest consensus state: current_term: 1 leader_uuid: "1122ffcba2914659812e449048dd8486" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.078548 16383 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:09:53.078900 16435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1122ffcba2914659812e449048dd8486" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.078966 16435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: This master's current role is: LEADER
I20251028 21:09:53.079043 16435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1122ffcba2914659812e449048dd8486" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.079100 16435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [sys.catalog]: This master's current role is: LEADER
I20251028 21:09:53.079445 16397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1122ffcba2914659812e449048dd8486" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.079542 16397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:09:53.079658 16383 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1122ffcba2914659812e449048dd8486" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580543d2bfd24c358e5dba831bd98f07" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 39915 } } peers { permanent_uuid: "1122ffcba2914659812e449048dd8486" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 40097 } } peers { permanent_uuid: "d2f0488d9b3c4c97b95c970d0e0bf8d2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 38153 } } }
I20251028 21:09:53.079715 16383 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:09:53.079865 16438 mvcc.cc:204] Tried to move back new op lower bound from 7215865008444678144 to 7215865008439263232. Current Snapshot: MvccSnapshot[applied={T|T < 7215865008444678144}]
I20251028 21:09:53.080221 16437 catalog_manager.cc:1357] Generated new cluster ID: 639d10a77ede4479880248877ee7c72c
I20251028 21:09:53.080281 16437 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:09:53.089957 16437 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:09:53.090397 16437 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:09:53.094574 16437 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486: Generated new TSK 0
I20251028 21:09:53.094743 16437 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:09:53.103739 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:53.103999 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:53.104182 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:09:53.104463 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:09:53.104501 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.104534 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:09:53.104591 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.108781 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:40911
I20251028 21:09:53.108843 16505 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:40911 every 8 connection(s)
I20251028 21:09:53.109236 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:53.111068 16513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:53.112466 16515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:53.112887 16507 heartbeater.cc:344] Connected to a master server at 127.15.205.62:39915
W20251028 21:09:53.112704 16512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:53.112968 16508 heartbeater.cc:344] Connected to a master server at 127.15.205.61:40097
I20251028 21:09:53.113018 16508 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.113094 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:53.113201 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251028 21:09:53.113247 16508 heartbeater.cc:507] Master 127.15.205.61:40097 requested a full tablet report, sending...
W20251028 21:09:53.113248 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:53.113338 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685793113338 us; error 0 us; skew 500 ppm
I20251028 21:09:53.112969 16507 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.113471 16507 heartbeater.cc:507] Master 127.15.205.62:39915 requested a full tablet report, sending...
I20251028 21:09:53.113636 16276 ts_manager.cc:194] Registered new tserver with Master: 2d19753cb4694ba6bb043e03d1a0e926 (127.15.205.1:40911)
I20251028 21:09:53.113729 16210 ts_manager.cc:194] Registered new tserver with Master: 2d19753cb4694ba6bb043e03d1a0e926 (127.15.205.1:40911)
I20251028 21:09:53.113924 16180 webserver.cc:492] Webserver started at http://127.15.205.2:33157/ using document root <none> and password file <none>
I20251028 21:09:53.114001 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:53.114037 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:53.114120 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:53.114401 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "2d2d863d395141c689f17e010b0fd78a"
format_stamp: "Formatted at 2025-10-28 21:09:53 on dist-test-slave-zmkx"
I20251028 21:09:53.114506 16276 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57266
I20251028 21:09:53.115532 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:53.116086 16520 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.116266 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251028 21:09:53.116317 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "2d2d863d395141c689f17e010b0fd78a"
format_stamp: "Formatted at 2025-10-28 21:09:53 on dist-test-slave-zmkx"
I20251028 21:09:53.116355 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:53.118057 16506 heartbeater.cc:344] Connected to a master server at 127.15.205.60:38153
I20251028 21:09:53.118103 16506 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.118145 16506 heartbeater.cc:507] Master 127.15.205.60:38153 requested a full tablet report, sending...
I20251028 21:09:53.118363 16342 ts_manager.cc:194] Registered new tserver with Master: 2d19753cb4694ba6bb043e03d1a0e926 (127.15.205.1:40911)
I20251028 21:09:53.125337 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:53.125535 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:53.125814 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:09:53.125993 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:09:53.126020 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.126049 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:09:53.126077 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.129117 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:34969
I20251028 21:09:53.129179 16583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:34969 every 8 connection(s)
I20251028 21:09:53.129534 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:53.132026 16591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:53.132584 16593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:53.132663 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:53.132653 16585 heartbeater.cc:344] Connected to a master server at 127.15.205.62:39915
I20251028 21:09:53.132843 16585 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.132937 16585 heartbeater.cc:507] Master 127.15.205.62:39915 requested a full tablet report, sending...
I20251028 21:09:53.133134 16210 ts_manager.cc:194] Registered new tserver with Master: 2d2d863d395141c689f17e010b0fd78a (127.15.205.2:34969)
W20251028 21:09:53.133373 16590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:53.133613 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:53.133648 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:53.133666 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685793133665 us; error 0 us; skew 500 ppm
I20251028 21:09:53.133725 16584 heartbeater.cc:344] Connected to a master server at 127.15.205.60:38153
I20251028 21:09:53.133771 16584 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.133850 16584 heartbeater.cc:507] Master 127.15.205.60:38153 requested a full tablet report, sending...
I20251028 21:09:53.134057 16342 ts_manager.cc:194] Registered new tserver with Master: 2d2d863d395141c689f17e010b0fd78a (127.15.205.2:34969)
I20251028 21:09:53.134491 16180 webserver.cc:492] Webserver started at http://127.15.205.3:46345/ using document root <none> and password file <none>
I20251028 21:09:53.134588 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:53.134646 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:53.134697 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:53.135003 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "f84f2075589f4f17a151d0029aab968f"
format_stamp: "Formatted at 2025-10-28 21:09:53 on dist-test-slave-zmkx"
I20251028 21:09:53.135385 16586 heartbeater.cc:344] Connected to a master server at 127.15.205.61:40097
I20251028 21:09:53.135437 16586 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.135529 16586 heartbeater.cc:507] Master 127.15.205.61:40097 requested a full tablet report, sending...
I20251028 21:09:53.135715 16276 ts_manager.cc:194] Registered new tserver with Master: 2d2d863d395141c689f17e010b0fd78a (127.15.205.2:34969)
I20251028 21:09:53.136094 16276 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57278
I20251028 21:09:53.139447 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20251028 21:09:53.139884 16599 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.140003 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.140048 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "f84f2075589f4f17a151d0029aab968f"
format_stamp: "Formatted at 2025-10-28 21:09:53 on dist-test-slave-zmkx"
I20251028 21:09:53.140089 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:53.159776 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:53.160000 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:53.160151 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:09:53.160342 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:09:53.160372 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.160403 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:09:53.160431 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:53.164252 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:37401
I20251028 21:09:53.164330 16662 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:37401 every 8 connection(s)
I20251028 21:09:53.167965 16664 heartbeater.cc:344] Connected to a master server at 127.15.205.62:39915
I20251028 21:09:53.168069 16664 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.168144 16665 heartbeater.cc:344] Connected to a master server at 127.15.205.61:40097
I20251028 21:09:53.168208 16665 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.168290 16664 heartbeater.cc:507] Master 127.15.205.62:39915 requested a full tablet report, sending...
I20251028 21:09:53.168313 16665 heartbeater.cc:507] Master 127.15.205.61:40097 requested a full tablet report, sending...
I20251028 21:09:53.168349 16663 heartbeater.cc:344] Connected to a master server at 127.15.205.60:38153
I20251028 21:09:53.168438 16663 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:53.168521 16663 heartbeater.cc:507] Master 127.15.205.60:38153 requested a full tablet report, sending...
I20251028 21:09:53.168584 16276 ts_manager.cc:194] Registered new tserver with Master: f84f2075589f4f17a151d0029aab968f (127.15.205.3:37401)
I20251028 21:09:53.168704 16342 ts_manager.cc:194] Registered new tserver with Master: f84f2075589f4f17a151d0029aab968f (127.15.205.3:37401)
I20251028 21:09:53.168704 16210 ts_manager.cc:194] Registered new tserver with Master: f84f2075589f4f17a151d0029aab968f (127.15.205.3:37401)
I20251028 21:09:53.168983 16276 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57284
I20251028 21:09:53.169445 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004956525s
I20251028 21:09:54.041996 16394 catalog_manager.cc:1269] Loaded cluster ID: 639d10a77ede4479880248877ee7c72c
I20251028 21:09:54.042069 16394 catalog_manager.cc:1562] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: loading cluster ID for follower catalog manager: success
I20251028 21:09:54.042738 16394 catalog_manager.cc:1584] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: acquiring CA information for follower catalog manager: success
I20251028 21:09:54.042881 16394 catalog_manager.cc:1612] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 21:09:54.044714 16408 catalog_manager.cc:1269] Loaded cluster ID: 639d10a77ede4479880248877ee7c72c
I20251028 21:09:54.044765 16408 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07: loading cluster ID for follower catalog manager: success
I20251028 21:09:54.045356 16408 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07: acquiring CA information for follower catalog manager: success
I20251028 21:09:54.045477 16408 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 21:09:54.115053 16508 heartbeater.cc:499] Master 127.15.205.61:40097 was elected leader, sending a full tablet report...
I20251028 21:09:54.136571 16586 heartbeater.cc:499] Master 127.15.205.61:40097 was elected leader, sending a full tablet report...
I20251028 21:09:54.169462 16665 heartbeater.cc:499] Master 127.15.205.61:40097 was elected leader, sending a full tablet report...
I20251028 21:09:55.068252 16420 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:09:55.220263 16180 test_util.cc:276] Using random seed: 1879550752
I20251028 21:09:55.227561 16276 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39718:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 21:09:55.234572 16469 tablet_service.cc:1505] Processing CreateTablet for tablet 56ff7e1874ec4596b9af13e9665606fa (DEFAULT_TABLE table=test-workload [id=87daa0bb2c674856aa954e41d232ee5b]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:09:55.234650 16627 tablet_service.cc:1505] Processing CreateTablet for tablet 50b4c6d7ab0446dbb5722562e2a8aadb (DEFAULT_TABLE table=test-workload [id=87daa0bb2c674856aa954e41d232ee5b]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:09:55.234690 16548 tablet_service.cc:1505] Processing CreateTablet for tablet cdaf3c70335e4b70815991db11c1adcb (DEFAULT_TABLE table=test-workload [id=87daa0bb2c674856aa954e41d232ee5b]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:09:55.234795 16548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdaf3c70335e4b70815991db11c1adcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:55.234568 16470 tablet_service.cc:1505] Processing CreateTablet for tablet 352a857ee5524f1a82299381dd9d33ab (DEFAULT_TABLE table=test-workload [id=87daa0bb2c674856aa954e41d232ee5b]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:09:55.234797 16469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ff7e1874ec4596b9af13e9665606fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:55.234797 16627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50b4c6d7ab0446dbb5722562e2a8aadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:55.234992 16470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 352a857ee5524f1a82299381dd9d33ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:55.236342 16700 tablet_bootstrap.cc:492] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f: Bootstrap starting.
I20251028 21:09:55.236353 16702 tablet_bootstrap.cc:492] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926: Bootstrap starting.
I20251028 21:09:55.236714 16702 tablet_bootstrap.cc:654] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:55.236850 16701 tablet_bootstrap.cc:492] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a: Bootstrap starting.
I20251028 21:09:55.237154 16701 tablet_bootstrap.cc:654] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:55.237198 16702 tablet_bootstrap.cc:492] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926: No bootstrap required, opened a new log
I20251028 21:09:55.237236 16702 ts_tablet_manager.cc:1403] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:55.236690 16700 tablet_bootstrap.cc:654] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:55.237356 16702 raft_consensus.cc:359] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } }
I20251028 21:09:55.237422 16702 raft_consensus.cc:385] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:55.237448 16702 raft_consensus.cc:740] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d19753cb4694ba6bb043e03d1a0e926, State: Initialized, Role: FOLLOWER
I20251028 21:09:55.237541 16702 consensus_queue.cc:260] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } }
I20251028 21:09:55.237591 16702 raft_consensus.cc:399] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:09:55.237617 16702 raft_consensus.cc:493] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:09:55.237665 16702 raft_consensus.cc:3060] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:55.237728 16701 tablet_bootstrap.cc:492] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a: No bootstrap required, opened a new log
I20251028 21:09:55.237805 16701 ts_tablet_manager.cc:1403] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:55.237847 16700 tablet_bootstrap.cc:492] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f: No bootstrap required, opened a new log
I20251028 21:09:55.237913 16700 ts_tablet_manager.cc:1403] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:09:55.237964 16701 raft_consensus.cc:359] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2d863d395141c689f17e010b0fd78a" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 34969 } }
I20251028 21:09:55.238018 16701 raft_consensus.cc:385] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:55.238018 16700 raft_consensus.cc:359] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84f2075589f4f17a151d0029aab968f" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 37401 } }
I20251028 21:09:55.238093 16701 raft_consensus.cc:740] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2d863d395141c689f17e010b0fd78a, State: Initialized, Role: FOLLOWER
I20251028 21:09:55.238138 16700 raft_consensus.cc:385] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:55.238184 16700 raft_consensus.cc:740] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84f2075589f4f17a151d0029aab968f, State: Initialized, Role: FOLLOWER
I20251028 21:09:55.238214 16701 consensus_queue.cc:260] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2d863d395141c689f17e010b0fd78a" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 34969 } }
I20251028 21:09:55.238255 16700 consensus_queue.cc:260] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84f2075589f4f17a151d0029aab968f" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 37401 } }
I20251028 21:09:55.238263 16702 raft_consensus.cc:515] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } }
I20251028 21:09:55.238345 16700 raft_consensus.cc:399] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:09:55.238325 16701 raft_consensus.cc:399] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:09:55.238368 16700 raft_consensus.cc:493] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:09:55.238374 16702 leader_election.cc:304] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d19753cb4694ba6bb043e03d1a0e926; no voters:
I20251028 21:09:55.238406 16700 raft_consensus.cc:3060] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:55.238375 16701 raft_consensus.cc:493] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:09:55.238519 16702 leader_election.cc:290] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:09:55.238528 16701 raft_consensus.cc:3060] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:55.238718 16702 ts_tablet_manager.cc:1434] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:55.238919 16702 tablet_bootstrap.cc:492] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926: Bootstrap starting.
I20251028 21:09:55.239086 16700 raft_consensus.cc:515] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84f2075589f4f17a151d0029aab968f" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 37401 } }
I20251028 21:09:55.239182 16700 leader_election.cc:304] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f84f2075589f4f17a151d0029aab968f; no voters:
I20251028 21:09:55.239250 16702 tablet_bootstrap.cc:654] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:55.238569 16706 raft_consensus.cc:2804] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:55.239317 16700 leader_election.cc:290] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:09:55.239390 16706 raft_consensus.cc:697] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 LEADER]: Becoming Leader. State: Replica: 2d19753cb4694ba6bb043e03d1a0e926, State: Running, Role: LEADER
I20251028 21:09:55.239298 16701 raft_consensus.cc:515] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2d863d395141c689f17e010b0fd78a" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 34969 } }
I20251028 21:09:55.239645 16708 raft_consensus.cc:2804] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:55.239661 16701 leader_election.cc:304] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2d863d395141c689f17e010b0fd78a; no voters:
I20251028 21:09:55.239732 16702 tablet_bootstrap.cc:492] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926: No bootstrap required, opened a new log
I20251028 21:09:55.239763 16708 raft_consensus.cc:697] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 1 LEADER]: Becoming Leader. State: Replica: f84f2075589f4f17a151d0029aab968f, State: Running, Role: LEADER
I20251028 21:09:55.239780 16702 ts_tablet_manager.cc:1403] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:55.239850 16708 consensus_queue.cc:237] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84f2075589f4f17a151d0029aab968f" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 37401 } }
I20251028 21:09:55.239921 16702 raft_consensus.cc:359] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } }
I20251028 21:09:55.239977 16702 raft_consensus.cc:385] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:55.240000 16702 raft_consensus.cc:740] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d19753cb4694ba6bb043e03d1a0e926, State: Initialized, Role: FOLLOWER
I20251028 21:09:55.240087 16702 consensus_queue.cc:260] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } }
I20251028 21:09:55.240156 16702 raft_consensus.cc:399] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:09:55.240181 16702 raft_consensus.cc:493] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:09:55.240199 16702 raft_consensus.cc:3060] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:55.240532 16276 catalog_manager.cc:5649] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f reported cstate change: term changed from 0 to 1, leader changed from <none> to f84f2075589f4f17a151d0029aab968f (127.15.205.3). New cstate: current_term: 1 leader_uuid: "f84f2075589f4f17a151d0029aab968f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84f2075589f4f17a151d0029aab968f" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 37401 } health_report { overall_health: HEALTHY } } }
I20251028 21:09:55.240994 16702 raft_consensus.cc:515] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } }
I20251028 21:09:55.241109 16702 leader_election.cc:304] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d19753cb4694ba6bb043e03d1a0e926; no voters:
I20251028 21:09:55.241222 16702 leader_election.cc:290] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:09:55.241387 16702 ts_tablet_manager.cc:1434] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:09:55.241505 16707 raft_consensus.cc:2804] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:55.241658 16707 raft_consensus.cc:697] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 LEADER]: Becoming Leader. State: Replica: 2d19753cb4694ba6bb043e03d1a0e926, State: Running, Role: LEADER
I20251028 21:09:55.239775 16701 leader_election.cc:290] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:09:55.239539 16700 ts_tablet_manager.cc:1434] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:09:55.239530 16706 consensus_queue.cc:237] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } }
I20251028 21:09:55.242101 16716 raft_consensus.cc:2804] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:55.241897 16707 consensus_queue.cc:237] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } }
I20251028 21:09:55.242235 16716 raft_consensus.cc:697] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 1 LEADER]: Becoming Leader. State: Replica: 2d2d863d395141c689f17e010b0fd78a, State: Running, Role: LEADER
I20251028 21:09:55.242287 16716 consensus_queue.cc:237] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2d863d395141c689f17e010b0fd78a" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 34969 } }
I20251028 21:09:55.242866 16275 catalog_manager.cc:5649] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d2d863d395141c689f17e010b0fd78a (127.15.205.2). New cstate: current_term: 1 leader_uuid: "2d2d863d395141c689f17e010b0fd78a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2d863d395141c689f17e010b0fd78a" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 34969 } health_report { overall_health: HEALTHY } } }
I20251028 21:09:55.243121 16274 catalog_manager.cc:5649] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d19753cb4694ba6bb043e03d1a0e926 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "2d19753cb4694ba6bb043e03d1a0e926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } health_report { overall_health: HEALTHY } } }
I20251028 21:09:55.242102 16701 ts_tablet_manager.cc:1434] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251028 21:09:55.245087 16275 catalog_manager.cc:5649] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d19753cb4694ba6bb043e03d1a0e926 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "2d19753cb4694ba6bb043e03d1a0e926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d19753cb4694ba6bb043e03d1a0e926" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40911 } health_report { overall_health: HEALTHY } } }
I20251028 21:09:55.247788 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:09:55.250286 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:09:55.250414 16180 tablet_replica.cc:333] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926: stopping tablet replica
I20251028 21:09:55.250492 16180 raft_consensus.cc:2243] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:09:55.250540 16180 raft_consensus.cc:2272] T 352a857ee5524f1a82299381dd9d33ab P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:09:55.250835 16180 tablet_replica.cc:333] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926: stopping tablet replica
I20251028 21:09:55.250875 16180 raft_consensus.cc:2243] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:09:55.250896 16180 raft_consensus.cc:2272] T 56ff7e1874ec4596b9af13e9665606fa P 2d19753cb4694ba6bb043e03d1a0e926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:09:55.252660 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:09:55.253465 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:09:55.256726 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:09:55.256858 16180 tablet_replica.cc:333] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a: stopping tablet replica
I20251028 21:09:55.256935 16180 raft_consensus.cc:2243] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:09:55.256979 16180 raft_consensus.cc:2272] T cdaf3c70335e4b70815991db11c1adcb P 2d2d863d395141c689f17e010b0fd78a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:09:55.258280 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:09:55.258823 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:09:55.261683 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:09:55.261834 16180 tablet_replica.cc:333] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f: stopping tablet replica
I20251028 21:09:55.261909 16180 raft_consensus.cc:2243] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:09:55.261945 16180 raft_consensus.cc:2272] T 50b4c6d7ab0446dbb5722562e2a8aadb P f84f2075589f4f17a151d0029aab968f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:09:55.273550 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:09:55.274186 16180 master.cc:561] Master@127.15.205.62:39915 shutting down...
I20251028 21:09:55.276394 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:09:55.276446 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:09:55.276468 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 580543d2bfd24c358e5dba831bd98f07: stopping tablet replica
I20251028 21:09:55.288030 16180 master.cc:583] Master@127.15.205.62:39915 shutdown complete.
I20251028 21:09:55.288895 16180 master.cc:561] Master@127.15.205.61:40097 shutting down...
I20251028 21:09:55.291256 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:09:55.291405 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:09:55.291441 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1122ffcba2914659812e449048dd8486: stopping tablet replica
I20251028 21:09:55.293234 16180 master.cc:583] Master@127.15.205.61:40097 shutdown complete.
I20251028 21:09:55.294052 16180 master.cc:561] Master@127.15.205.60:38153 shutting down...
I20251028 21:09:55.295877 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:09:55.295933 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:09:55.295956 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P d2f0488d9b3c4c97b95c970d0e0bf8d2: stopping tablet replica
I20251028 21:09:55.307595 16180 master.cc:583] Master@127.15.205.60:38153 shutdown complete.
[ OK ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing (2363 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn
I20251028 21:09:55.312486 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:37607
I20251028 21:09:55.312633 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:55.313617 16728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:55.313730 16726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:55.313695 16725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:55.313788 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:55.313908 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:55.313973 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:55.313997 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685795313996 us; error 0 us; skew 500 ppm
I20251028 21:09:55.314508 16180 webserver.cc:492] Webserver started at http://127.15.205.62:37571/ using document root <none> and password file <none>
I20251028 21:09:55.314585 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:55.314630 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:55.314684 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:55.314930 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "87a29b1d525a4b629554ca9e92195203"
format_stamp: "Formatted at 2025-10-28 21:09:55 on dist-test-slave-zmkx"
I20251028 21:09:55.315749 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:55.316139 16733 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.316316 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.316354 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "87a29b1d525a4b629554ca9e92195203"
format_stamp: "Formatted at 2025-10-28 21:09:55 on dist-test-slave-zmkx"
I20251028 21:09:55.316401 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:55.338749 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:55.338901 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:55.341166 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:37607
I20251028 21:09:55.343829 16785 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:37607 every 8 connection(s)
I20251028 21:09:55.343896 16786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:55.344799 16786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203: Bootstrap starting.
I20251028 21:09:55.345064 16786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:55.345496 16786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203: No bootstrap required, opened a new log
I20251028 21:09:55.345605 16786 raft_consensus.cc:359] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a29b1d525a4b629554ca9e92195203" member_type: VOTER }
I20251028 21:09:55.345651 16786 raft_consensus.cc:385] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:55.345674 16786 raft_consensus.cc:740] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a29b1d525a4b629554ca9e92195203, State: Initialized, Role: FOLLOWER
I20251028 21:09:55.345723 16786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a29b1d525a4b629554ca9e92195203" member_type: VOTER }
I20251028 21:09:55.345762 16786 raft_consensus.cc:399] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:09:55.345783 16786 raft_consensus.cc:493] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:09:55.345815 16786 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:55.346261 16786 raft_consensus.cc:515] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a29b1d525a4b629554ca9e92195203" member_type: VOTER }
I20251028 21:09:55.346313 16786 leader_election.cc:304] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87a29b1d525a4b629554ca9e92195203; no voters:
I20251028 21:09:55.346415 16786 leader_election.cc:290] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:09:55.346457 16789 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:55.346541 16786 sys_catalog.cc:565] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:09:55.346616 16789 raft_consensus.cc:697] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 1 LEADER]: Becoming Leader. State: Replica: 87a29b1d525a4b629554ca9e92195203, State: Running, Role: LEADER
I20251028 21:09:55.346669 16789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a29b1d525a4b629554ca9e92195203" member_type: VOTER }
I20251028 21:09:55.346824 16789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87a29b1d525a4b629554ca9e92195203. Latest consensus state: current_term: 1 leader_uuid: "87a29b1d525a4b629554ca9e92195203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a29b1d525a4b629554ca9e92195203" member_type: VOTER } }
I20251028 21:09:55.346879 16789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [sys.catalog]: This master's current role is: LEADER
I20251028 21:09:55.347056 16794 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:09:55.346861 16790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87a29b1d525a4b629554ca9e92195203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a29b1d525a4b629554ca9e92195203" member_type: VOTER } }
I20251028 21:09:55.347230 16794 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:09:55.347321 16790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [sys.catalog]: This master's current role is: LEADER
I20251028 21:09:55.347893 16794 catalog_manager.cc:1357] Generated new cluster ID: 287d0df7643e4a2d82678857ee95119f
I20251028 21:09:55.347899 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:09:55.347949 16794 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:09:55.354230 16794 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:09:55.354595 16794 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:09:55.360891 16794 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203: Generated new TSK 0
I20251028 21:09:55.360956 16794 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:09:55.363517 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:55.364773 16808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:55.364778 16807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:55.364899 16810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:55.365069 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:55.365180 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:55.365211 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:55.365227 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685795365227 us; error 0 us; skew 500 ppm
I20251028 21:09:55.366003 16180 webserver.cc:492] Webserver started at http://127.15.205.1:40281/ using document root <none> and password file <none>
I20251028 21:09:55.366076 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:55.366109 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:55.366158 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:55.366392 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "ca6c43008b3c4d6491238aabe57b7def"
format_stamp: "Formatted at 2025-10-28 21:09:55 on dist-test-slave-zmkx"
I20251028 21:09:55.367434 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:55.367935 16815 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.368045 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.368088 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "ca6c43008b3c4d6491238aabe57b7def"
format_stamp: "Formatted at 2025-10-28 21:09:55 on dist-test-slave-zmkx"
I20251028 21:09:55.368132 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:55.387111 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:55.387318 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:55.387431 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:09:55.387579 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:09:55.387603 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.387620 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:09:55.387646 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.390027 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:38931
I20251028 21:09:55.390060 16878 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:38931 every 8 connection(s)
I20251028 21:09:55.390316 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:55.391278 16884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:55.391292 16883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:55.391381 16886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:55.391633 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:55.391744 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:55.391788 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:55.391816 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685795391817 us; error 0 us; skew 500 ppm
I20251028 21:09:55.392313 16180 webserver.cc:492] Webserver started at http://127.15.205.2:34811/ using document root <none> and password file <none>
I20251028 21:09:55.392386 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:55.392436 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:55.392484 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:55.392689 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "99eb2565e1624ffda8c3548100fc776c"
format_stamp: "Formatted at 2025-10-28 21:09:55 on dist-test-slave-zmkx"
I20251028 21:09:55.393564 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:55.393882 16879 heartbeater.cc:344] Connected to a master server at 127.15.205.62:37607
I20251028 21:09:55.393942 16879 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:55.394039 16879 heartbeater.cc:507] Master 127.15.205.62:37607 requested a full tablet report, sending...
I20251028 21:09:55.394068 16891 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.394160 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.394204 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "99eb2565e1624ffda8c3548100fc776c"
format_stamp: "Formatted at 2025-10-28 21:09:55 on dist-test-slave-zmkx"
I20251028 21:09:55.394240 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:55.394279 16750 ts_manager.cc:194] Registered new tserver with Master: ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931)
I20251028 21:09:55.394845 16750 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53646
I20251028 21:09:55.418397 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:55.418586 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:55.418675 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:09:55.418826 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:09:55.418856 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.418875 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:09:55.418887 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.421391 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:37799
I20251028 21:09:55.421453 16954 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:37799 every 8 connection(s)
I20251028 21:09:55.421730 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:55.422904 16958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:55.422910 16959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:55.423066 16961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:55.423113 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:55.423236 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:55.423285 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:55.423302 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685795423302 us; error 0 us; skew 500 ppm
I20251028 21:09:55.423825 16180 webserver.cc:492] Webserver started at http://127.15.205.3:44659/ using document root <none> and password file <none>
I20251028 21:09:55.423918 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:55.423964 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:55.424011 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:55.424305 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "50ee98a33d714e7b8957480f5ed20986"
format_stamp: "Formatted at 2025-10-28 21:09:55 on dist-test-slave-zmkx"
I20251028 21:09:55.424682 16955 heartbeater.cc:344] Connected to a master server at 127.15.205.62:37607
I20251028 21:09:55.424738 16955 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:55.424830 16955 heartbeater.cc:507] Master 127.15.205.62:37607 requested a full tablet report, sending...
I20251028 21:09:55.425050 16750 ts_manager.cc:194] Registered new tserver with Master: 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799)
I20251028 21:09:55.425323 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:55.425479 16750 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53658
I20251028 21:09:55.425806 16966 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.425930 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.425984 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "50ee98a33d714e7b8957480f5ed20986"
format_stamp: "Formatted at 2025-10-28 21:09:55 on dist-test-slave-zmkx"
I20251028 21:09:55.426043 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:55.443702 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:55.443851 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:55.443948 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:09:55.444072 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:09:55.444092 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.444118 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:09:55.444133 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:55.447026 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:45045
I20251028 21:09:55.447100 17029 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:45045 every 8 connection(s)
I20251028 21:09:55.449661 17030 heartbeater.cc:344] Connected to a master server at 127.15.205.62:37607
I20251028 21:09:55.449705 17030 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:55.449792 17030 heartbeater.cc:507] Master 127.15.205.62:37607 requested a full tablet report, sending...
I20251028 21:09:55.450001 16750 ts_manager.cc:194] Registered new tserver with Master: 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:55.450294 16750 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53664
I20251028 21:09:55.450480 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00327395s
I20251028 21:09:56.395328 16879 heartbeater.cc:499] Master 127.15.205.62:37607 was elected leader, sending a full tablet report...
I20251028 21:09:56.425994 16955 heartbeater.cc:499] Master 127.15.205.62:37607 was elected leader, sending a full tablet report...
I20251028 21:09:56.450695 17030 heartbeater.cc:499] Master 127.15.205.62:37607 was elected leader, sending a full tablet report...
I20251028 21:09:56.475481 16180 test_util.cc:276] Using random seed: 1880805976
I20251028 21:09:56.480005 16750 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53670:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o""\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:09:56.480142 16750 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:09:56.484900 16913 tablet_service.cc:1505] Processing CreateTablet for tablet 79a3e264b1ed4c2085a39cc5d9d557c8 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251028 21:09:56.484934 16919 tablet_service.cc:1505] Processing CreateTablet for tablet 07b5269afdfb44e9a8c9bef615e40035 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251028 21:09:56.484906 16912 tablet_service.cc:1505] Processing CreateTablet for tablet 2c2d284d3d704f1e9a9491a5c174c3e8 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251028 21:09:56.485042 16913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a3e264b1ed4c2085a39cc5d9d557c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.485090 16918 tablet_service.cc:1505] Processing CreateTablet for tablet 0c18886b173f49a480c460d29af19886 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251028 21:09:56.485143 16912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2d284d3d704f1e9a9491a5c174c3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.485193 16918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c18886b173f49a480c460d29af19886. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.486339 17050 tablet_bootstrap.cc:492] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c: Bootstrap starting.
I20251028 21:09:56.486382 16916 tablet_service.cc:1505] Processing CreateTablet for tablet 80c08544f68c48fe8b468d77b0b38188 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251028 21:09:56.486457 16916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c08544f68c48fe8b468d77b0b38188. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.486763 17050 tablet_bootstrap.cc:654] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.487289 16843 tablet_service.cc:1505] Processing CreateTablet for tablet 07b5269afdfb44e9a8c9bef615e40035 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251028 21:09:56.487316 16842 tablet_service.cc:1505] Processing CreateTablet for tablet 0c18886b173f49a480c460d29af19886 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251028 21:09:56.487339 16915 tablet_service.cc:1505] Processing CreateTablet for tablet 53fa0467f182447fa7ce88cc635d9316 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251028 21:09:56.487393 16843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07b5269afdfb44e9a8c9bef615e40035. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.487448 16914 tablet_service.cc:1505] Processing CreateTablet for tablet 6b05743ef9914c0c9c70bf7f99312c20 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251028 21:09:56.487476 16842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c18886b173f49a480c460d29af19886. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.486348 16917 tablet_service.cc:1505] Processing CreateTablet for tablet bf44a146569c4a30be37e954067a01fd (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251028 21:09:56.487403 16915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53fa0467f182447fa7ce88cc635d9316. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.488682 16841 tablet_service.cc:1505] Processing CreateTablet for tablet bf44a146569c4a30be37e954067a01fd (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251028 21:09:56.488770 16841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf44a146569c4a30be37e954067a01fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.488884 17052 tablet_bootstrap.cc:492] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def: Bootstrap starting.
I20251028 21:09:56.488988 16994 tablet_service.cc:1505] Processing CreateTablet for tablet 07b5269afdfb44e9a8c9bef615e40035 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251028 21:09:56.489084 16994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07b5269afdfb44e9a8c9bef615e40035. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.489946 17052 tablet_bootstrap.cc:654] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.490094 16840 tablet_service.cc:1505] Processing CreateTablet for tablet 80c08544f68c48fe8b468d77b0b38188 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251028 21:09:56.490196 16840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c08544f68c48fe8b468d77b0b38188. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.490244 16993 tablet_service.cc:1505] Processing CreateTablet for tablet 0c18886b173f49a480c460d29af19886 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251028 21:09:56.490324 16993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c18886b173f49a480c460d29af19886. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.490808 17052 tablet_bootstrap.cc:492] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def: No bootstrap required, opened a new log
I20251028 21:09:56.490952 17052 ts_tablet_manager.cc:1403] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:09:56.491207 17052 raft_consensus.cc:359] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.491372 17052 raft_consensus.cc:385] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.491396 16987 tablet_service.cc:1505] Processing CreateTablet for tablet 2c2d284d3d704f1e9a9491a5c174c3e8 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251028 21:09:56.491403 17052 raft_consensus.cc:740] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.491503 16987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2d284d3d704f1e9a9491a5c174c3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.491492 17052 consensus_queue.cc:260] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.491573 16838 tablet_service.cc:1505] Processing CreateTablet for tablet 6b05743ef9914c0c9c70bf7f99312c20 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251028 21:09:56.491658 16838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b05743ef9914c0c9c70bf7f99312c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.491727 17052 ts_tablet_manager.cc:1434] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.491786 17052 tablet_bootstrap.cc:492] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def: Bootstrap starting.
I20251028 21:09:56.492190 17052 tablet_bootstrap.cc:654] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.492738 16988 tablet_service.cc:1505] Processing CreateTablet for tablet 79a3e264b1ed4c2085a39cc5d9d557c8 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251028 21:09:56.492771 17052 tablet_bootstrap.cc:492] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def: No bootstrap required, opened a new log
I20251028 21:09:56.492808 17052 ts_tablet_manager.cc:1403] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.492862 16989 tablet_service.cc:1505] Processing CreateTablet for tablet 6b05743ef9914c0c9c70bf7f99312c20 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251028 21:09:56.492995 16990 tablet_service.cc:1505] Processing CreateTablet for tablet 53fa0467f182447fa7ce88cc635d9316 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251028 21:09:56.492950 17052 raft_consensus.cc:359] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.493019 16836 tablet_service.cc:1505] Processing CreateTablet for tablet 2c2d284d3d704f1e9a9491a5c174c3e8 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251028 21:09:56.493033 17052 raft_consensus.cc:385] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.493073 17052 raft_consensus.cc:740] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.493108 17052 consensus_queue.cc:260] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.493276 17052 ts_tablet_manager.cc:1434] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.493333 17052 tablet_bootstrap.cc:492] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def: Bootstrap starting.
I20251028 21:09:56.493109 16836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2d284d3d704f1e9a9491a5c174c3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.493625 17053 tablet_bootstrap.cc:492] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986: Bootstrap starting.
I20251028 21:09:56.493860 17052 tablet_bootstrap.cc:654] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.493996 17050 tablet_bootstrap.cc:492] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c: No bootstrap required, opened a new log
I20251028 21:09:56.494045 17050 ts_tablet_manager.cc:1403] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20251028 21:09:56.494204 17050 raft_consensus.cc:359] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.494277 17050 raft_consensus.cc:385] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.494306 17050 raft_consensus.cc:740] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.494355 17050 consensus_queue.cc:260] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.494513 17050 ts_tablet_manager.cc:1434] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:09:56.494565 17050 tablet_bootstrap.cc:492] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c: Bootstrap starting.
I20251028 21:09:56.494655 17052 tablet_bootstrap.cc:492] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def: No bootstrap required, opened a new log
I20251028 21:09:56.494742 17052 ts_tablet_manager.cc:1403] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.494935 17052 raft_consensus.cc:359] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.492957 16837 tablet_service.cc:1505] Processing CreateTablet for tablet 79a3e264b1ed4c2085a39cc5d9d557c8 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251028 21:09:56.492812 16988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a3e264b1ed4c2085a39cc5d9d557c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.493151 16992 tablet_service.cc:1505] Processing CreateTablet for tablet bf44a146569c4a30be37e954067a01fd (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251028 21:09:56.495208 16992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf44a146569c4a30be37e954067a01fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.491534 16839 tablet_service.cc:1505] Processing CreateTablet for tablet 53fa0467f182447fa7ce88cc635d9316 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251028 21:09:56.493112 16991 tablet_service.cc:1505] Processing CreateTablet for tablet 80c08544f68c48fe8b468d77b0b38188 (DEFAULT_TABLE table=test-workload [id=0a36770df2a54dc5b589c4c93361fdce]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251028 21:09:56.495405 16990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53fa0467f182447fa7ce88cc635d9316. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.495764 16991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80c08544f68c48fe8b468d77b0b38188. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.488771 16914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b05743ef9914c0c9c70bf7f99312c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.496698 16917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf44a146569c4a30be37e954067a01fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.496812 16919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07b5269afdfb44e9a8c9bef615e40035. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.495127 16837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a3e264b1ed4c2085a39cc5d9d557c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.498066 17053 tablet_bootstrap.cc:654] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.498128 16839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53fa0467f182447fa7ce88cc635d9316. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.495024 17052 raft_consensus.cc:385] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.498404 17052 raft_consensus.cc:740] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.498467 17052 consensus_queue.cc:260] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.498647 17050 tablet_bootstrap.cc:654] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.498793 17053 tablet_bootstrap.cc:492] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986: No bootstrap required, opened a new log
I20251028 21:09:56.498845 17053 ts_tablet_manager.cc:1403] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20251028 21:09:56.498981 17053 raft_consensus.cc:359] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.499038 17053 raft_consensus.cc:385] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.499075 17053 raft_consensus.cc:740] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.499112 17053 consensus_queue.cc:260] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.499248 17053 ts_tablet_manager.cc:1434] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:09:56.499326 17053 tablet_bootstrap.cc:492] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986: Bootstrap starting.
I20251028 21:09:56.499428 17052 ts_tablet_manager.cc:1434] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def: Time spent starting tablet: real 0.005s user 0.000s sys 0.000s
I20251028 21:09:56.498131 16989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b05743ef9914c0c9c70bf7f99312c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:09:56.499935 17053 tablet_bootstrap.cc:654] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.500344 17052 tablet_bootstrap.cc:492] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def: Bootstrap starting.
I20251028 21:09:56.500483 17053 tablet_bootstrap.cc:492] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986: No bootstrap required, opened a new log
I20251028 21:09:56.500525 17053 ts_tablet_manager.cc:1403] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:56.500705 17053 raft_consensus.cc:359] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.500764 17052 tablet_bootstrap.cc:654] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.500775 17053 raft_consensus.cc:385] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.500872 17053 raft_consensus.cc:740] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.500936 17053 consensus_queue.cc:260] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.500981 17050 tablet_bootstrap.cc:492] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c: No bootstrap required, opened a new log
I20251028 21:09:56.501024 17050 ts_tablet_manager.cc:1403] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20251028 21:09:56.501035 17053 ts_tablet_manager.cc:1434] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251028 21:09:56.501132 17053 tablet_bootstrap.cc:492] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986: Bootstrap starting.
I20251028 21:09:56.501190 17050 raft_consensus.cc:359] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.501253 17050 raft_consensus.cc:385] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.501273 17050 raft_consensus.cc:740] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.501322 17050 consensus_queue.cc:260] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.501421 17050 ts_tablet_manager.cc:1434] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.501482 17050 tablet_bootstrap.cc:492] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c: Bootstrap starting.
I20251028 21:09:56.501545 17053 tablet_bootstrap.cc:654] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.501830 17050 tablet_bootstrap.cc:654] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.502362 17050 tablet_bootstrap.cc:492] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c: No bootstrap required, opened a new log
I20251028 21:09:56.502409 17050 ts_tablet_manager.cc:1403] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.502456 17053 tablet_bootstrap.cc:492] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986: No bootstrap required, opened a new log
I20251028 21:09:56.502506 17053 ts_tablet_manager.cc:1403] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:56.502542 17050 raft_consensus.cc:359] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.502583 17050 raft_consensus.cc:385] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.502599 17050 raft_consensus.cc:740] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.502627 17053 raft_consensus.cc:359] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.502645 17050 consensus_queue.cc:260] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.502751 17050 ts_tablet_manager.cc:1434] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.502816 17050 tablet_bootstrap.cc:492] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c: Bootstrap starting.
I20251028 21:09:56.502923 17052 tablet_bootstrap.cc:492] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def: No bootstrap required, opened a new log
I20251028 21:09:56.502970 17052 ts_tablet_manager.cc:1403] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:09:56.503111 17052 raft_consensus.cc:359] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.503186 17052 raft_consensus.cc:385] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.503217 17052 raft_consensus.cc:740] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.503230 17053 raft_consensus.cc:385] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.503221 17050 tablet_bootstrap.cc:654] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.503341 17053 raft_consensus.cc:740] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.503404 17053 consensus_queue.cc:260] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.503495 17053 ts_tablet_manager.cc:1434] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:09:56.503585 17053 tablet_bootstrap.cc:492] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986: Bootstrap starting.
I20251028 21:09:56.503284 17052 consensus_queue.cc:260] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.503661 17052 ts_tablet_manager.cc:1434] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.503712 17052 tablet_bootstrap.cc:492] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def: Bootstrap starting.
I20251028 21:09:56.504091 17052 tablet_bootstrap.cc:654] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.504264 17053 tablet_bootstrap.cc:654] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.504375 17050 tablet_bootstrap.cc:492] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c: No bootstrap required, opened a new log
I20251028 21:09:56.504421 17050 ts_tablet_manager.cc:1403] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:09:56.504570 17050 raft_consensus.cc:359] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.504608 17050 raft_consensus.cc:385] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.504621 17050 raft_consensus.cc:740] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.504626 17052 tablet_bootstrap.cc:492] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def: No bootstrap required, opened a new log
I20251028 21:09:56.504666 17050 consensus_queue.cc:260] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.504667 17052 ts_tablet_manager.cc:1403] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.504762 17050 ts_tablet_manager.cc:1434] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.504807 17050 tablet_bootstrap.cc:492] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c: Bootstrap starting.
I20251028 21:09:56.504863 17052 raft_consensus.cc:359] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.504923 17052 raft_consensus.cc:385] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.504913 17053 tablet_bootstrap.cc:492] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986: No bootstrap required, opened a new log
I20251028 21:09:56.504966 17052 raft_consensus.cc:740] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.505007 17053 ts_tablet_manager.cc:1403] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:56.505025 17052 consensus_queue.cc:260] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.505105 17052 ts_tablet_manager.cc:1434] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.505146 17052 tablet_bootstrap.cc:492] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def: Bootstrap starting.
I20251028 21:09:56.505146 17053 raft_consensus.cc:359] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.505239 17053 raft_consensus.cc:385] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.505265 17053 raft_consensus.cc:740] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.505182 17050 tablet_bootstrap.cc:654] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.505544 17052 tablet_bootstrap.cc:654] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.505592 17053 consensus_queue.cc:260] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.505694 17053 ts_tablet_manager.cc:1434] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:09:56.505937 17053 tablet_bootstrap.cc:492] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986: Bootstrap starting.
I20251028 21:09:56.506081 17052 tablet_bootstrap.cc:492] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def: No bootstrap required, opened a new log
I20251028 21:09:56.506124 17052 ts_tablet_manager.cc:1403] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.506274 17052 raft_consensus.cc:359] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.506335 17052 raft_consensus.cc:385] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.506356 17052 raft_consensus.cc:740] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.506373 17053 tablet_bootstrap.cc:654] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.506397 17052 consensus_queue.cc:260] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.506479 17050 tablet_bootstrap.cc:492] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c: No bootstrap required, opened a new log
I20251028 21:09:56.506532 17050 ts_tablet_manager.cc:1403] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:09:56.506670 17050 raft_consensus.cc:359] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.506729 17050 raft_consensus.cc:385] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.506754 17050 raft_consensus.cc:740] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.506804 17050 consensus_queue.cc:260] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.506889 17050 ts_tablet_manager.cc:1434] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.506909 17053 tablet_bootstrap.cc:492] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986: No bootstrap required, opened a new log
I20251028 21:09:56.506484 17052 ts_tablet_manager.cc:1434] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:09:56.506956 17053 ts_tablet_manager.cc:1403] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:56.506985 17052 tablet_bootstrap.cc:492] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def: Bootstrap starting.
I20251028 21:09:56.507100 17053 raft_consensus.cc:359] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.507158 17053 raft_consensus.cc:385] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.507196 17053 raft_consensus.cc:740] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.507246 17053 consensus_queue.cc:260] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.507292 17050 tablet_bootstrap.cc:492] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c: Bootstrap starting.
I20251028 21:09:56.507412 17052 tablet_bootstrap.cc:654] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.507663 17050 tablet_bootstrap.cc:654] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.507663 17053 ts_tablet_manager.cc:1434] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.507867 17053 tablet_bootstrap.cc:492] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986: Bootstrap starting.
I20251028 21:09:56.507923 17052 tablet_bootstrap.cc:492] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def: No bootstrap required, opened a new log
I20251028 21:09:56.507963 17052 ts_tablet_manager.cc:1403] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.508085 17052 raft_consensus.cc:359] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.508145 17052 raft_consensus.cc:385] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.508164 17052 raft_consensus.cc:740] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.508203 17052 consensus_queue.cc:260] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.508260 17053 tablet_bootstrap.cc:654] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.508287 17052 ts_tablet_manager.cc:1434] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.508347 17052 tablet_bootstrap.cc:492] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def: Bootstrap starting.
I20251028 21:09:56.508373 17050 tablet_bootstrap.cc:492] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c: No bootstrap required, opened a new log
I20251028 21:09:56.508401 17050 ts_tablet_manager.cc:1403] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.508535 17050 raft_consensus.cc:359] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.508584 17050 raft_consensus.cc:385] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.508601 17050 raft_consensus.cc:740] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.508654 17050 consensus_queue.cc:260] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.508705 17052 tablet_bootstrap.cc:654] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.508740 17050 ts_tablet_manager.cc:1434] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.508791 17050 tablet_bootstrap.cc:492] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c: Bootstrap starting.
I20251028 21:09:56.508832 17053 tablet_bootstrap.cc:492] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986: No bootstrap required, opened a new log
I20251028 21:09:56.508873 17053 ts_tablet_manager.cc:1403] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.509006 17053 raft_consensus.cc:359] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.509053 17053 raft_consensus.cc:385] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.509076 17053 raft_consensus.cc:740] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.509116 17053 consensus_queue.cc:260] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.509191 17050 tablet_bootstrap.cc:654] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.509483 17053 ts_tablet_manager.cc:1434] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:09:56.509706 17053 tablet_bootstrap.cc:492] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986: Bootstrap starting.
I20251028 21:09:56.509694 17052 tablet_bootstrap.cc:492] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def: No bootstrap required, opened a new log
I20251028 21:09:56.509779 17052 ts_tablet_manager.cc:1403] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.509691 17050 tablet_bootstrap.cc:492] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c: No bootstrap required, opened a new log
I20251028 21:09:56.509919 17050 ts_tablet_manager.cc:1403] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.509935 17052 raft_consensus.cc:359] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.509994 17052 raft_consensus.cc:385] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.510017 17052 raft_consensus.cc:740] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.510057 17050 raft_consensus.cc:359] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.510105 17053 tablet_bootstrap.cc:654] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.510129 17050 raft_consensus.cc:385] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.510154 17050 raft_consensus.cc:740] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.510208 17050 consensus_queue.cc:260] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.510074 17052 consensus_queue.cc:260] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.510318 17052 ts_tablet_manager.cc:1434] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.510515 17050 ts_tablet_manager.cc:1434] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:09:56.510726 17050 tablet_bootstrap.cc:492] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c: Bootstrap starting.
I20251028 21:09:56.510938 17053 tablet_bootstrap.cc:492] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986: No bootstrap required, opened a new log
I20251028 21:09:56.510986 17053 ts_tablet_manager.cc:1403] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.511148 17053 raft_consensus.cc:359] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.511206 17053 raft_consensus.cc:385] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.511220 17053 raft_consensus.cc:740] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.511286 17053 consensus_queue.cc:260] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.511307 17050 tablet_bootstrap.cc:654] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.511404 17053 ts_tablet_manager.cc:1434] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.511466 17053 tablet_bootstrap.cc:492] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986: Bootstrap starting.
I20251028 21:09:56.511811 17053 tablet_bootstrap.cc:654] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986: Neither blocks nor log segments found. Creating new log.
I20251028 21:09:56.512053 17050 tablet_bootstrap.cc:492] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c: No bootstrap required, opened a new log
I20251028 21:09:56.512115 17050 ts_tablet_manager.cc:1403] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:09:56.512244 17050 raft_consensus.cc:359] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.512305 17050 raft_consensus.cc:385] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.512327 17050 raft_consensus.cc:740] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.512377 17050 consensus_queue.cc:260] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.512398 17053 tablet_bootstrap.cc:492] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986: No bootstrap required, opened a new log
I20251028 21:09:56.512447 17053 ts_tablet_manager.cc:1403] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:56.512466 17050 ts_tablet_manager.cc:1434] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.512576 17053 raft_consensus.cc:359] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.512630 17053 raft_consensus.cc:385] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:09:56.512646 17053 raft_consensus.cc:740] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Initialized, Role: FOLLOWER
I20251028 21:09:56.512692 17053 consensus_queue.cc:260] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.512778 17053 ts_tablet_manager.cc:1434] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.522883 17058 raft_consensus.cc:493] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:56.522948 17058 raft_consensus.cc:515] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.523118 17058 leader_election.cc:290] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.525451 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b05743ef9914c0c9c70bf7f99312c20" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c43008b3c4d6491238aabe57b7def" is_pre_election: true
I20251028 21:09:56.525519 16853 raft_consensus.cc:2468] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 0.
I20251028 21:09:56.525609 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b05743ef9914c0c9c70bf7f99312c20" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986" is_pre_election: true
I20251028 21:09:56.525662 17004 raft_consensus.cc:2468] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 0.
I20251028 21:09:56.525669 16893 leader_election.cc:304] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99eb2565e1624ffda8c3548100fc776c, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.525746 17058 raft_consensus.cc:2804] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:56.525775 17058 raft_consensus.cc:493] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:56.525795 17058 raft_consensus.cc:3060] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.526328 17058 raft_consensus.cc:515] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.526428 17058 leader_election.cc:290] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 1 election: Requested vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.526551 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b05743ef9914c0c9c70bf7f99312c20" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986"
I20251028 21:09:56.526547 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b05743ef9914c0c9c70bf7f99312c20" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c43008b3c4d6491238aabe57b7def"
I20251028 21:09:56.526638 17004 raft_consensus.cc:3060] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.526640 16853 raft_consensus.cc:3060] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.527302 17004 raft_consensus.cc:2468] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 1.
I20251028 21:09:56.527340 16853 raft_consensus.cc:2468] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 1.
I20251028 21:09:56.527443 16895 leader_election.cc:304] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, 99eb2565e1624ffda8c3548100fc776c; no voters:
I20251028 21:09:56.527542 17058 raft_consensus.cc:2804] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:56.527667 17058 raft_consensus.cc:697] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 1 LEADER]: Becoming Leader. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Running, Role: LEADER
I20251028 21:09:56.527720 17058 consensus_queue.cc:237] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.528141 16749 catalog_manager.cc:5649] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c reported cstate change: term changed from 0 to 1, leader changed from <none> to 99eb2565e1624ffda8c3548100fc776c (127.15.205.2). New cstate: current_term: 1 leader_uuid: "99eb2565e1624ffda8c3548100fc776c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: UNKNOWN } } }
I20251028 21:09:56.540594 17055 raft_consensus.cc:493] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:56.540642 17055 raft_consensus.cc:515] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.540750 17055 leader_election.cc:290] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.542987 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07b5269afdfb44e9a8c9bef615e40035" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986" is_pre_election: true
I20251028 21:09:56.543044 17004 raft_consensus.cc:2468] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 0.
I20251028 21:09:56.543054 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07b5269afdfb44e9a8c9bef615e40035" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c" is_pre_election: true
I20251028 21:09:56.543130 16929 raft_consensus.cc:2468] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 0.
I20251028 21:09:56.543190 16819 leader_election.cc:304] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.543278 17055 raft_consensus.cc:2804] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:56.543320 17055 raft_consensus.cc:493] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:56.543347 17055 raft_consensus.cc:3060] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.543787 17055 raft_consensus.cc:515] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.543895 17055 leader_election.cc:290] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 election: Requested vote from peers 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.544000 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07b5269afdfb44e9a8c9bef615e40035" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986"
I20251028 21:09:56.544047 17004 raft_consensus.cc:3060] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.544049 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07b5269afdfb44e9a8c9bef615e40035" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c"
I20251028 21:09:56.544133 16929 raft_consensus.cc:3060] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.544494 17004 raft_consensus.cc:2468] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 1.
I20251028 21:09:56.544612 16929 raft_consensus.cc:2468] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 1.
I20251028 21:09:56.544624 16819 leader_election.cc:304] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.544734 17055 raft_consensus.cc:2804] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:56.544822 17055 raft_consensus.cc:697] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Becoming Leader. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Running, Role: LEADER
I20251028 21:09:56.544855 17055 consensus_queue.cc:237] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } }
I20251028 21:09:56.545308 16749 catalog_manager.cc:5649] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def reported cstate change: term changed from 0 to 1, leader changed from <none> to ca6c43008b3c4d6491238aabe57b7def (127.15.205.1). New cstate: current_term: 1 leader_uuid: "ca6c43008b3c4d6491238aabe57b7def" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: HEALTHY } } }
I20251028 21:09:56.572456 17061 raft_consensus.cc:493] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:56.572510 17061 raft_consensus.cc:515] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.572664 17061 leader_election.cc:290] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799)
I20251028 21:09:56.574942 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf44a146569c4a30be37e954067a01fd" candidate_uuid: "50ee98a33d714e7b8957480f5ed20986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c" is_pre_election: true
I20251028 21:09:56.574965 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf44a146569c4a30be37e954067a01fd" candidate_uuid: "50ee98a33d714e7b8957480f5ed20986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c43008b3c4d6491238aabe57b7def" is_pre_election: true
I20251028 21:09:56.575002 16929 raft_consensus.cc:2468] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50ee98a33d714e7b8957480f5ed20986 in term 0.
I20251028 21:09:56.575038 16853 raft_consensus.cc:2468] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50ee98a33d714e7b8957480f5ed20986 in term 0.
I20251028 21:09:56.575139 16967 leader_election.cc:304] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, 99eb2565e1624ffda8c3548100fc776c; no voters:
I20251028 21:09:56.575229 17061 raft_consensus.cc:2804] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:56.575284 17061 raft_consensus.cc:493] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:56.575304 17061 raft_consensus.cc:3060] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.575717 17061 raft_consensus.cc:515] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.575812 17061 leader_election.cc:290] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [CANDIDATE]: Term 1 election: Requested vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799)
I20251028 21:09:56.575984 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf44a146569c4a30be37e954067a01fd" candidate_uuid: "50ee98a33d714e7b8957480f5ed20986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c"
I20251028 21:09:56.575994 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf44a146569c4a30be37e954067a01fd" candidate_uuid: "50ee98a33d714e7b8957480f5ed20986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c43008b3c4d6491238aabe57b7def"
I20251028 21:09:56.576038 16929 raft_consensus.cc:3060] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.576047 16853 raft_consensus.cc:3060] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.576515 16929 raft_consensus.cc:2468] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ee98a33d714e7b8957480f5ed20986 in term 1.
I20251028 21:09:56.576517 16853 raft_consensus.cc:2468] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ee98a33d714e7b8957480f5ed20986 in term 1.
I20251028 21:09:56.576714 16968 leader_election.cc:304] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.576797 17061 raft_consensus.cc:2804] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:56.576925 17061 raft_consensus.cc:697] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 1 LEADER]: Becoming Leader. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Running, Role: LEADER
I20251028 21:09:56.576988 17061 consensus_queue.cc:237] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.577421 16749 catalog_manager.cc:5649] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "50ee98a33d714e7b8957480f5ed20986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: HEALTHY } } }
I20251028 21:09:56.604138 17055 raft_consensus.cc:493] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:56.604202 17055 raft_consensus.cc:515] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.604321 17055 leader_election.cc:290] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.604475 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79a3e264b1ed4c2085a39cc5d9d557c8" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986" is_pre_election: true
I20251028 21:09:56.604529 17004 raft_consensus.cc:2468] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 0.
I20251028 21:09:56.604482 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79a3e264b1ed4c2085a39cc5d9d557c8" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c" is_pre_election: true
I20251028 21:09:56.604560 16929 raft_consensus.cc:2468] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 0.
I20251028 21:09:56.604645 16819 leader_election.cc:304] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.604732 17055 raft_consensus.cc:2804] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:56.604768 17055 raft_consensus.cc:493] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:56.604781 17055 raft_consensus.cc:3060] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.605201 17055 raft_consensus.cc:515] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.605290 17055 leader_election.cc:290] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 election: Requested vote from peers 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.605427 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79a3e264b1ed4c2085a39cc5d9d557c8" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986"
I20251028 21:09:56.605449 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79a3e264b1ed4c2085a39cc5d9d557c8" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c"
I20251028 21:09:56.605472 17004 raft_consensus.cc:3060] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.605487 16929 raft_consensus.cc:3060] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.605922 16929 raft_consensus.cc:2468] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 1.
I20251028 21:09:56.605937 17004 raft_consensus.cc:2468] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 1.
I20251028 21:09:56.606055 16819 leader_election.cc:304] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.606138 17055 raft_consensus.cc:2804] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:56.606185 17055 raft_consensus.cc:697] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Becoming Leader. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Running, Role: LEADER
I20251028 21:09:56.606235 17055 consensus_queue.cc:237] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.606647 16749 catalog_manager.cc:5649] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def reported cstate change: term changed from 0 to 1, leader changed from <none> to ca6c43008b3c4d6491238aabe57b7def (127.15.205.1). New cstate: current_term: 1 leader_uuid: "ca6c43008b3c4d6491238aabe57b7def" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: UNKNOWN } } }
W20251028 21:09:56.641232 16880 tablet.cc:2378] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251028 21:09:56.656409 17055 raft_consensus.cc:493] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:56.656462 17055 raft_consensus.cc:515] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.656589 17055 leader_election.cc:290] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.656738 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0c18886b173f49a480c460d29af19886" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c" is_pre_election: true
I20251028 21:09:56.656774 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0c18886b173f49a480c460d29af19886" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986" is_pre_election: true
I20251028 21:09:56.656829 16929 raft_consensus.cc:2468] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 0.
I20251028 21:09:56.656836 17004 raft_consensus.cc:2468] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 0.
I20251028 21:09:56.656967 16816 leader_election.cc:304] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99eb2565e1624ffda8c3548100fc776c, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.657099 17055 raft_consensus.cc:2804] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:56.657147 17055 raft_consensus.cc:493] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:56.657160 17055 raft_consensus.cc:3060] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.657567 17055 raft_consensus.cc:515] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.657663 17055 leader_election.cc:290] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 election: Requested vote from peers 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.657780 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0c18886b173f49a480c460d29af19886" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986"
I20251028 21:09:56.657781 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0c18886b173f49a480c460d29af19886" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c"
I20251028 21:09:56.657846 17004 raft_consensus.cc:3060] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.657857 16929 raft_consensus.cc:3060] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.658231 16929 raft_consensus.cc:2468] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 1.
I20251028 21:09:56.658241 17004 raft_consensus.cc:2468] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 1.
I20251028 21:09:56.658378 16819 leader_election.cc:304] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.658458 17055 raft_consensus.cc:2804] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:56.658509 17055 raft_consensus.cc:697] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Becoming Leader. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Running, Role: LEADER
I20251028 21:09:56.658560 17055 consensus_queue.cc:237] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.658947 16749 catalog_manager.cc:5649] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def reported cstate change: term changed from 0 to 1, leader changed from <none> to ca6c43008b3c4d6491238aabe57b7def (127.15.205.1). New cstate: current_term: 1 leader_uuid: "ca6c43008b3c4d6491238aabe57b7def" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: UNKNOWN } } }
I20251028 21:09:56.674801 17055 raft_consensus.cc:493] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:56.674854 17055 raft_consensus.cc:515] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.674955 17055 leader_election.cc:290] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.675083 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c2d284d3d704f1e9a9491a5c174c3e8" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c" is_pre_election: true
I20251028 21:09:56.675083 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c2d284d3d704f1e9a9491a5c174c3e8" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986" is_pre_election: true
I20251028 21:09:56.675181 16929 raft_consensus.cc:2468] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 0.
I20251028 21:09:56.675186 17004 raft_consensus.cc:2468] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 0.
I20251028 21:09:56.675314 16819 leader_election.cc:304] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.675377 17055 raft_consensus.cc:2804] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:56.675424 17055 raft_consensus.cc:493] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:56.675451 17055 raft_consensus.cc:3060] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.675839 17055 raft_consensus.cc:515] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.675942 17055 leader_election.cc:290] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 election: Requested vote from peers 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.676038 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c2d284d3d704f1e9a9491a5c174c3e8" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986"
I20251028 21:09:56.676091 17004 raft_consensus.cc:3060] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.676050 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2c2d284d3d704f1e9a9491a5c174c3e8" candidate_uuid: "ca6c43008b3c4d6491238aabe57b7def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c"
I20251028 21:09:56.676172 16929 raft_consensus.cc:3060] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.676625 16929 raft_consensus.cc:2468] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 1.
I20251028 21:09:56.676630 17004 raft_consensus.cc:2468] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca6c43008b3c4d6491238aabe57b7def in term 1.
I20251028 21:09:56.676779 16819 leader_election.cc:304] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.676859 17055 raft_consensus.cc:2804] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:56.676913 17055 raft_consensus.cc:697] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Becoming Leader. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Running, Role: LEADER
I20251028 21:09:56.676945 17055 consensus_queue.cc:237] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.677376 16749 catalog_manager.cc:5649] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def reported cstate change: term changed from 0 to 1, leader changed from <none> to ca6c43008b3c4d6491238aabe57b7def (127.15.205.1). New cstate: current_term: 1 leader_uuid: "ca6c43008b3c4d6491238aabe57b7def" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: UNKNOWN } } }
I20251028 21:09:56.772725 17061 raft_consensus.cc:493] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:56.772783 17061 raft_consensus.cc:515] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.772868 17061 leader_election.cc:290] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799)
I20251028 21:09:56.773036 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "53fa0467f182447fa7ce88cc635d9316" candidate_uuid: "50ee98a33d714e7b8957480f5ed20986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c" is_pre_election: true
I20251028 21:09:56.773063 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "53fa0467f182447fa7ce88cc635d9316" candidate_uuid: "50ee98a33d714e7b8957480f5ed20986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c43008b3c4d6491238aabe57b7def" is_pre_election: true
I20251028 21:09:56.773101 16929 raft_consensus.cc:2468] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50ee98a33d714e7b8957480f5ed20986 in term 0.
I20251028 21:09:56.773108 16853 raft_consensus.cc:2468] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50ee98a33d714e7b8957480f5ed20986 in term 0.
I20251028 21:09:56.773263 16968 leader_election.cc:304] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.773357 17061 raft_consensus.cc:2804] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:56.773401 17061 raft_consensus.cc:493] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:56.773415 17061 raft_consensus.cc:3060] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.773890 17061 raft_consensus.cc:515] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.773990 17061 leader_election.cc:290] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [CANDIDATE]: Term 1 election: Requested vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 99eb2565e1624ffda8c3548100fc776c (127.15.205.2:37799)
I20251028 21:09:56.774091 16929 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "53fa0467f182447fa7ce88cc635d9316" candidate_uuid: "50ee98a33d714e7b8957480f5ed20986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99eb2565e1624ffda8c3548100fc776c"
I20251028 21:09:56.774091 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "53fa0467f182447fa7ce88cc635d9316" candidate_uuid: "50ee98a33d714e7b8957480f5ed20986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c43008b3c4d6491238aabe57b7def"
I20251028 21:09:56.774166 16929 raft_consensus.cc:3060] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.774179 16853 raft_consensus.cc:3060] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.774660 16853 raft_consensus.cc:2468] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ee98a33d714e7b8957480f5ed20986 in term 1.
I20251028 21:09:56.774660 16929 raft_consensus.cc:2468] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ee98a33d714e7b8957480f5ed20986 in term 1.
I20251028 21:09:56.774850 16967 leader_election.cc:304] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, 99eb2565e1624ffda8c3548100fc776c; no voters:
I20251028 21:09:56.774935 17061 raft_consensus.cc:2804] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:56.774989 17061 raft_consensus.cc:697] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 1 LEADER]: Becoming Leader. State: Replica: 50ee98a33d714e7b8957480f5ed20986, State: Running, Role: LEADER
I20251028 21:09:56.775033 17061 consensus_queue.cc:237] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.775557 16749 catalog_manager.cc:5649] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "50ee98a33d714e7b8957480f5ed20986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: HEALTHY } } }
I20251028 21:09:56.862764 17058 raft_consensus.cc:493] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:09:56.862852 17058 raft_consensus.cc:515] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.862953 17058 leader_election.cc:290] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.863088 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80c08544f68c48fe8b468d77b0b38188" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c43008b3c4d6491238aabe57b7def" is_pre_election: true
I20251028 21:09:56.863147 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80c08544f68c48fe8b468d77b0b38188" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986" is_pre_election: true
I20251028 21:09:56.863170 16853 raft_consensus.cc:2468] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 0.
I20251028 21:09:56.863197 17004 raft_consensus.cc:2468] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 0.
I20251028 21:09:56.863319 16895 leader_election.cc:304] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, 99eb2565e1624ffda8c3548100fc776c; no voters:
I20251028 21:09:56.863389 17058 raft_consensus.cc:2804] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:09:56.863427 17058 raft_consensus.cc:493] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:09:56.863447 17058 raft_consensus.cc:3060] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.863919 17058 raft_consensus.cc:515] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.864013 17058 leader_election.cc:290] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 1 election: Requested vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:56.864153 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80c08544f68c48fe8b468d77b0b38188" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6c43008b3c4d6491238aabe57b7def"
I20251028 21:09:56.864226 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "80c08544f68c48fe8b468d77b0b38188" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ee98a33d714e7b8957480f5ed20986"
I20251028 21:09:56.864229 16853 raft_consensus.cc:3060] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.864295 17004 raft_consensus.cc:3060] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:09:56.864822 17004 raft_consensus.cc:2468] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 1.
I20251028 21:09:56.864822 16853 raft_consensus.cc:2468] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 1.
I20251028 21:09:56.864986 16893 leader_election.cc:304] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99eb2565e1624ffda8c3548100fc776c, ca6c43008b3c4d6491238aabe57b7def; no voters:
I20251028 21:09:56.865072 17058 raft_consensus.cc:2804] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:09:56.865121 17058 raft_consensus.cc:697] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 1 LEADER]: Becoming Leader. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Running, Role: LEADER
I20251028 21:09:56.865168 17058 consensus_queue.cc:237] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:56.865698 16749 catalog_manager.cc:5649] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c reported cstate change: term changed from 0 to 1, leader changed from <none> to 99eb2565e1624ffda8c3548100fc776c (127.15.205.2). New cstate: current_term: 1 leader_uuid: "99eb2565e1624ffda8c3548100fc776c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: UNKNOWN } } }
I20251028 21:09:56.919726 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:09:56.920966 17097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:56.921044 17100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:09:56.921103 17098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:09:56.921281 16180 server_base.cc:1047] running on GCE node
I20251028 21:09:56.921416 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:09:56.921448 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:09:56.921473 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685796921472 us; error 0 us; skew 500 ppm
I20251028 21:09:56.922156 16180 webserver.cc:492] Webserver started at http://127.15.205.4:33795/ using document root <none> and password file <none>
I20251028 21:09:56.922238 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:09:56.922283 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:09:56.922346 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:09:56.922684 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-3-root/instance:
uuid: "5bada290ee4e4c0c8e270169bef28697"
format_stamp: "Formatted at 2025-10-28 21:09:56 on dist-test-slave-zmkx"
I20251028 21:09:56.923583 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:09:56.924003 17105 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.924127 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.924170 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-3-root
uuid: "5bada290ee4e4c0c8e270169bef28697"
format_stamp: "Formatted at 2025-10-28 21:09:56 on dist-test-slave-zmkx"
I20251028 21:09:56.924206 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1761685792944951-16180-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:09:56.933663 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:09:56.934043 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:09:56.934165 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:09:56.934317 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:09:56.934342 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.934360 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:09:56.934387 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:09:56.937770 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.4:38165
I20251028 21:09:56.937809 17168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.4:38165 every 8 connection(s)
I20251028 21:09:56.940563 17169 heartbeater.cc:344] Connected to a master server at 127.15.205.62:37607
I20251028 21:09:56.940608 17169 heartbeater.cc:461] Registering TS with master...
I20251028 21:09:56.940685 17169 heartbeater.cc:507] Master 127.15.205.62:37607 requested a full tablet report, sending...
I20251028 21:09:56.940876 16749 ts_manager.cc:194] Registered new tserver with Master: 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4:38165)
I20251028 21:09:56.941192 16749 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53682
I20251028 21:09:56.944959 17058 consensus_queue.cc:1048] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:56.946171 17058 consensus_queue.cc:1048] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:56.957775 17061 consensus_queue.cc:1048] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:56.958837 17178 consensus_queue.cc:1048] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:56.971318 17055 consensus_queue.cc:1048] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:56.972517 17181 consensus_queue.cc:1048] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.005378 17181 consensus_queue.cc:1048] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.006614 17181 consensus_queue.cc:1048] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.045270 17181 consensus_queue.cc:1048] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.046227 17086 consensus_queue.cc:1048] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.069082 17086 consensus_queue.cc:1048] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.070108 17055 consensus_queue.cc:1048] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.171573 17178 consensus_queue.cc:1048] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.172621 17090 consensus_queue.cc:1048] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.277935 17058 consensus_queue.cc:1048] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.279243 17192 consensus_queue.cc:1048] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:09:57.347896 16805 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:09:57.350607 16853 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "79a3e264b1ed4c2085a39cc5d9d557c8"
dest_uuid: "ca6c43008b3c4d6491238aabe57b7def"
mode: GRACEFUL
new_leader_uuid: "99eb2565e1624ffda8c3548100fc776c"
from {username='slave'} at 127.0.0.1:39408
I20251028 21:09:57.350692 16853 raft_consensus.cc:606] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Received request to transfer leadership to TS 99eb2565e1624ffda8c3548100fc776c
I20251028 21:09:57.350863 16805 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:09:57.350929 16805 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:09:57.495361 17055 raft_consensus.cc:993] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def: : Instructing follower 99eb2565e1624ffda8c3548100fc776c to start an election
I20251028 21:09:57.495450 17055 raft_consensus.cc:1081] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Signalling peer 99eb2565e1624ffda8c3548100fc776c to start an election
I20251028 21:09:57.495635 16929 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "79a3e264b1ed4c2085a39cc5d9d557c8"
dest_uuid: "99eb2565e1624ffda8c3548100fc776c"
from {username='slave'} at 127.0.0.1:52724
I20251028 21:09:57.495698 16929 raft_consensus.cc:493] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:09:57.495719 16929 raft_consensus.cc:3060] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:09:57.496268 16929 raft_consensus.cc:515] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:57.496379 16929 leader_election.cc:290] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 2 election: Requested vote from peers ca6c43008b3c4d6491238aabe57b7def (127.15.205.1:38931), 50ee98a33d714e7b8957480f5ed20986 (127.15.205.3:45045)
I20251028 21:09:57.496512 16853 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79a3e264b1ed4c2085a39cc5d9d557c8" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca6c43008b3c4d6491238aabe57b7def"
I20251028 21:09:57.496574 16853 raft_consensus.cc:3055] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:09:57.496591 16853 raft_consensus.cc:740] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca6c43008b3c4d6491238aabe57b7def, State: Running, Role: LEADER
I20251028 21:09:57.496523 17004 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "79a3e264b1ed4c2085a39cc5d9d557c8" candidate_uuid: "99eb2565e1624ffda8c3548100fc776c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50ee98a33d714e7b8957480f5ed20986"
I20251028 21:09:57.496629 16853 consensus_queue.cc:260] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:57.496628 17004 raft_consensus.cc:3060] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:09:57.496716 16853 raft_consensus.cc:3060] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:09:57.497176 17004 raft_consensus.cc:2468] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 2.
I20251028 21:09:57.497217 16853 raft_consensus.cc:2468] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99eb2565e1624ffda8c3548100fc776c in term 2.
I20251028 21:09:57.497321 16895 leader_election.cc:304] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ee98a33d714e7b8957480f5ed20986, 99eb2565e1624ffda8c3548100fc776c; no voters:
I20251028 21:09:57.497414 17058 raft_consensus.cc:2804] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:09:57.497473 17058 raft_consensus.cc:697] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 2 LEADER]: Becoming Leader. State: Replica: 99eb2565e1624ffda8c3548100fc776c, State: Running, Role: LEADER
I20251028 21:09:57.497519 17058 consensus_queue.cc:237] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } }
I20251028 21:09:57.497893 16750 catalog_manager.cc:5649] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c reported cstate change: term changed from 1 to 2, leader changed from ca6c43008b3c4d6491238aabe57b7def (127.15.205.1) to 99eb2565e1624ffda8c3548100fc776c (127.15.205.2). New cstate: current_term: 2 leader_uuid: "99eb2565e1624ffda8c3548100fc776c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } health_report { overall_health: UNKNOWN } } }
I20251028 21:09:57.918825 16853 raft_consensus.cc:1275] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 2 FOLLOWER]: Refusing update from remote peer 99eb2565e1624ffda8c3548100fc776c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251028 21:09:57.919041 17082 consensus_queue.cc:1048] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:09:57.919839 17004 raft_consensus.cc:1275] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 2 FOLLOWER]: Refusing update from remote peer 99eb2565e1624ffda8c3548100fc776c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251028 21:09:57.920094 17082 consensus_queue.cc:1048] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:09:57.941619 17169 heartbeater.cc:499] Master 127.15.205.62:37607 was elected leader, sending a full tablet report...
I20251028 21:09:59.351215 16805 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:09:59.351486 16805 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:09:59.351536 16805 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:01.351686 16805 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:01.351951 16805 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:01.351999 16805 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:02.354496 17003 consensus_queue.cc:237] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } attrs { replace: true } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } }
I20251028 21:10:02.355037 16929 raft_consensus.cc:1275] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Refusing update from remote peer 50ee98a33d714e7b8957480f5ed20986: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:02.355180 16853 raft_consensus.cc:1275] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Refusing update from remote peer 50ee98a33d714e7b8957480f5ed20986: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:02.355332 17061 consensus_queue.cc:1048] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:02.355535 17206 consensus_queue.cc:1048] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:02.356053 17207 raft_consensus.cc:2955] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } attrs { replace: true } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.356141 16929 raft_consensus.cc:2955] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } attrs { replace: true } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.357215 16749 catalog_manager.cc:5649] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c reported cstate change: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "50ee98a33d714e7b8957480f5ed20986" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } attrs { replace: true } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.357303 16853 raft_consensus.cc:2955] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } attrs { replace: true } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
W20251028 21:10:02.358588 16968 consensus_peers.cc:597] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 -> Peer 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4:38165): Couldn't send request to peer 5bada290ee4e4c0c8e270169bef28697. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf44a146569c4a30be37e954067a01fd. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:10:02.360214 16853 consensus_queue.cc:237] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } attrs { replace: true } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } }
I20251028 21:10:02.360651 16929 raft_consensus.cc:1275] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Refusing update from remote peer ca6c43008b3c4d6491238aabe57b7def: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:02.360688 17003 raft_consensus.cc:1275] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Refusing update from remote peer ca6c43008b3c4d6491238aabe57b7def: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:02.360808 17219 consensus_queue.cc:1048] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:02.360896 17218 consensus_queue.cc:1048] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:02.361537 17218 raft_consensus.cc:2955] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } attrs { replace: true } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.361657 16929 raft_consensus.cc:2955] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } attrs { replace: true } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.361788 17003 raft_consensus.cc:2955] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } attrs { replace: true } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.362643 16748 catalog_manager.cc:5649] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def reported cstate change: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "ca6c43008b3c4d6491238aabe57b7def" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:10:02.363641 17004 consensus_queue.cc:237] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } }
I20251028 21:10:02.364137 16853 raft_consensus.cc:1275] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Refusing update from remote peer 50ee98a33d714e7b8957480f5ed20986: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251028 21:10:02.364179 16817 consensus_peers.cc:597] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def -> Peer 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4:38165): Couldn't send request to peer 5bada290ee4e4c0c8e270169bef28697. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c2d284d3d704f1e9a9491a5c174c3e8. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:10:02.364228 16929 raft_consensus.cc:1275] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Refusing update from remote peer 50ee98a33d714e7b8957480f5ed20986: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251028 21:10:02.364238 16968 consensus_peers.cc:597] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 -> Peer 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4:38165): Couldn't send request to peer 5bada290ee4e4c0c8e270169bef28697. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53fa0467f182447fa7ce88cc635d9316. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:10:02.364387 17207 consensus_queue.cc:1048] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:02.364506 17207 consensus_queue.cc:1048] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:02.364933 17061 raft_consensus.cc:2955] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.365046 16853 raft_consensus.cc:2955] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.365123 16929 raft_consensus.cc:2955] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.365943 16748 catalog_manager.cc:5649] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def reported cstate change: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "50ee98a33d714e7b8957480f5ed20986" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.366108 16853 consensus_queue.cc:237] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } }
I20251028 21:10:02.366554 16929 raft_consensus.cc:1275] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Refusing update from remote peer ca6c43008b3c4d6491238aabe57b7def: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:02.366642 17004 raft_consensus.cc:1275] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Refusing update from remote peer ca6c43008b3c4d6491238aabe57b7def: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251028 21:10:02.366775 16817 consensus_peers.cc:597] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def -> Peer 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4:38165): Couldn't send request to peer 5bada290ee4e4c0c8e270169bef28697. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c18886b173f49a480c460d29af19886. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:10:02.366775 17181 consensus_queue.cc:1048] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:02.366863 17218 consensus_queue.cc:1048] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:02.367470 17218 raft_consensus.cc:2955] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.367646 16929 raft_consensus.cc:2955] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.367651 17004 raft_consensus.cc:2955] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.368340 16749 catalog_manager.cc:5649] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c reported cstate change: config changed from index -1 to 2, NON_VOTER 5bada290ee4e4c0c8e270169bef28697 (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "ca6c43008b3c4d6491238aabe57b7def" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ca6c43008b3c4d6491238aabe57b7def" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38931 } } peers { permanent_uuid: "99eb2565e1624ffda8c3548100fc776c" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 37799 } attrs { replace: true } } peers { permanent_uuid: "50ee98a33d714e7b8957480f5ed20986" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45045 } } peers { permanent_uuid: "5bada290ee4e4c0c8e270169bef28697" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 38165 } attrs { promote: true } } }
I20251028 21:10:02.452468 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
W20251028 21:10:02.453581 16735 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20251028 21:10:02.454046 16799 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:10:02.454105 16799 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:10:02.455348 16799 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:10:02.455394 16799 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251028 21:10:02.456040 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:02.456121 16180 tablet_replica.cc:333] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def: stopping tablet replica
I20251028 21:10:02.456187 16180 raft_consensus.cc:2243] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.456229 16180 raft_consensus.cc:2272] T 79a3e264b1ed4c2085a39cc5d9d557c8 P ca6c43008b3c4d6491238aabe57b7def [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.456380 16180 tablet_replica.cc:333] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def: stopping tablet replica
I20251028 21:10:02.456413 16180 raft_consensus.cc:2243] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.456465 16180 raft_consensus.cc:2272] T 07b5269afdfb44e9a8c9bef615e40035 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.456562 16180 tablet_replica.cc:333] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def: stopping tablet replica
I20251028 21:10:02.456593 16180 raft_consensus.cc:2243] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.456669 16180 raft_consensus.cc:2272] T 2c2d284d3d704f1e9a9491a5c174c3e8 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.456836 16180 tablet_replica.cc:333] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def: stopping tablet replica
I20251028 21:10:02.456877 16180 raft_consensus.cc:2243] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.456940 16180 raft_consensus.cc:2272] T bf44a146569c4a30be37e954067a01fd P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.457142 16180 tablet_replica.cc:333] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def: stopping tablet replica
I20251028 21:10:02.457199 16180 raft_consensus.cc:2243] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.457262 16180 raft_consensus.cc:2272] T 0c18886b173f49a480c460d29af19886 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.457427 16180 tablet_replica.cc:333] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def: stopping tablet replica
I20251028 21:10:02.457470 16180 raft_consensus.cc:2243] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.457508 16180 raft_consensus.cc:2272] T 80c08544f68c48fe8b468d77b0b38188 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.457615 16180 tablet_replica.cc:333] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def: stopping tablet replica
I20251028 21:10:02.457651 16180 raft_consensus.cc:2243] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.457690 16180 raft_consensus.cc:2272] T 53fa0467f182447fa7ce88cc635d9316 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.457859 16180 tablet_replica.cc:333] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def: stopping tablet replica
I20251028 21:10:02.457897 16180 raft_consensus.cc:2243] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.457931 16180 raft_consensus.cc:2272] T 6b05743ef9914c0c9c70bf7f99312c20 P ca6c43008b3c4d6491238aabe57b7def [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.470826 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:10:02.472080 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:10:02.476323 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:02.476438 16180 tablet_replica.cc:333] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c: stopping tablet replica
I20251028 21:10:02.476495 16180 raft_consensus.cc:2243] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.476557 16180 raft_consensus.cc:2272] T 07b5269afdfb44e9a8c9bef615e40035 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.476696 16180 tablet_replica.cc:333] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c: stopping tablet replica
I20251028 21:10:02.476743 16180 raft_consensus.cc:2243] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.476819 16180 raft_consensus.cc:2272] T bf44a146569c4a30be37e954067a01fd P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.476991 16180 tablet_replica.cc:333] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c: stopping tablet replica
I20251028 21:10:02.477057 16180 raft_consensus.cc:2243] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.477097 16180 raft_consensus.cc:2272] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.477269 16180 tablet_replica.cc:333] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c: stopping tablet replica
I20251028 21:10:02.477319 16180 raft_consensus.cc:2243] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.477368 16180 raft_consensus.cc:2272] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 99eb2565e1624ffda8c3548100fc776c [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.477474 16180 tablet_replica.cc:333] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c: stopping tablet replica
I20251028 21:10:02.477514 16180 raft_consensus.cc:2243] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.477548 16180 raft_consensus.cc:2272] T 0c18886b173f49a480c460d29af19886 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.477694 16180 tablet_replica.cc:333] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c: stopping tablet replica
I20251028 21:10:02.477780 16180 raft_consensus.cc:2243] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.477830 16180 raft_consensus.cc:2272] T 80c08544f68c48fe8b468d77b0b38188 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.477958 16180 tablet_replica.cc:333] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c: stopping tablet replica
I20251028 21:10:02.477996 16180 raft_consensus.cc:2243] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.478037 16180 raft_consensus.cc:2272] T 53fa0467f182447fa7ce88cc635d9316 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.478196 16180 tablet_replica.cc:333] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c: stopping tablet replica
I20251028 21:10:02.478235 16180 raft_consensus.cc:2243] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.478286 16180 raft_consensus.cc:2272] T 6b05743ef9914c0c9c70bf7f99312c20 P 99eb2565e1624ffda8c3548100fc776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.479867 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:10:02.480911 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
W20251028 21:10:02.481479 16799 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:10:02.481534 16799 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:10:02.482038 16799 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:10:02.482086 16799 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251028 21:10:02.484495 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:02.484607 16180 tablet_replica.cc:333] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986: stopping tablet replica
I20251028 21:10:02.484670 16180 raft_consensus.cc:2243] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.484709 16180 raft_consensus.cc:2272] T 80c08544f68c48fe8b468d77b0b38188 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.484809 16180 tablet_replica.cc:333] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986: stopping tablet replica
I20251028 21:10:02.484840 16180 raft_consensus.cc:2243] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.484861 16180 raft_consensus.cc:2272] T 0c18886b173f49a480c460d29af19886 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.485044 16180 tablet_replica.cc:333] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986: stopping tablet replica
I20251028 21:10:02.485080 16180 raft_consensus.cc:2243] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.485105 16180 raft_consensus.cc:2272] T 07b5269afdfb44e9a8c9bef615e40035 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.485178 16180 tablet_replica.cc:333] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986: stopping tablet replica
I20251028 21:10:02.485201 16180 raft_consensus.cc:2243] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.485224 16180 raft_consensus.cc:2272] T 2c2d284d3d704f1e9a9491a5c174c3e8 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.485347 16180 tablet_replica.cc:333] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986: stopping tablet replica
I20251028 21:10:02.485375 16180 raft_consensus.cc:2243] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.485392 16180 raft_consensus.cc:2272] T 79a3e264b1ed4c2085a39cc5d9d557c8 P 50ee98a33d714e7b8957480f5ed20986 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.485455 16180 tablet_replica.cc:333] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986: stopping tablet replica
I20251028 21:10:02.485477 16180 raft_consensus.cc:2243] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:02.485497 16180 raft_consensus.cc:2272] T 6b05743ef9914c0c9c70bf7f99312c20 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.485554 16180 tablet_replica.cc:333] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986: stopping tablet replica
I20251028 21:10:02.485575 16180 raft_consensus.cc:2243] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.485616 16180 raft_consensus.cc:2272] T 53fa0467f182447fa7ce88cc635d9316 P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.485785 16180 tablet_replica.cc:333] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986: stopping tablet replica
I20251028 21:10:02.485817 16180 raft_consensus.cc:2243] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.485895 16180 raft_consensus.cc:2272] T bf44a146569c4a30be37e954067a01fd P 50ee98a33d714e7b8957480f5ed20986 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.487627 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:10:02.488512 16180 tablet_server.cc:178] TabletServer@127.15.205.4:0 shutting down...
I20251028 21:10:02.491664 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:02.502807 16180 tablet_server.cc:195] TabletServer@127.15.205.4:0 shutdown complete.
I20251028 21:10:02.503510 16180 master.cc:561] Master@127.15.205.62:37607 shutting down...
I20251028 21:10:02.505861 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:02.505940 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:02.505982 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 87a29b1d525a4b629554ca9e92195203: stopping tablet replica
I20251028 21:10:02.517445 16180 master.cc:583] Master@127.15.205.62:37607 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn (7212 ms)
[ RUN ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing
I20251028 21:10:02.525087 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:36989,127.15.205.61:41801,127.15.205.60:43203
I20251028 21:10:02.525228 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:02.526443 17233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.526512 17231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.526609 17230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.526714 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:02.526795 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:02.526878 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:02.526906 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685802526905 us; error 0 us; skew 500 ppm
I20251028 21:10:02.527495 16180 webserver.cc:492] Webserver started at http://127.15.205.62:46209/ using document root <none> and password file <none>
I20251028 21:10:02.527557 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:02.527599 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:02.527643 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:02.527892 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "516e00ec9b3642e182a2b2bf08ff76b0"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.528698 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:10:02.529090 17238 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.529201 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.529243 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "516e00ec9b3642e182a2b2bf08ff76b0"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.529289 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:02.545987 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:02.546156 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:02.548501 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:36989
I20251028 21:10:02.549014 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:10:02.553494 17290 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:36989 every 8 connection(s)
I20251028 21:10:02.553580 17291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 21:10:02.554250 17296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.554287 17294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.554332 17291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
W20251028 21:10:02.554287 17293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.555256 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:02.555383 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:02.555414 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:02.555440 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685802555439 us; error 0 us; skew 500 ppm
I20251028 21:10:02.555969 16180 webserver.cc:492] Webserver started at http://127.15.205.61:37121/ using document root <none> and password file <none>
I20251028 21:10:02.556058 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:02.556104 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:02.556152 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:02.556398 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root/instance:
uuid: "fc891714934a4ab78f8f7be7a9c799ae"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.557317 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:02.557735 17303 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.557860 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.557910 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root
uuid: "fc891714934a4ab78f8f7be7a9c799ae"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.558189 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:02.558736 17291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } has no permanent_uuid. Determining permanent_uuid...
W20251028 21:10:02.559228 17291 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.205.61:41801: Network error: Client connection negotiation failed: client connection to 127.15.205.61:41801: connect: Connection refused (error 111)
I20251028 21:10:02.567453 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:02.567602 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:02.569809 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.61:41801
I20251028 21:10:02.569832 17356 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.61:41801 every 8 connection(s)
I20251028 21:10:02.570050 17357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:02.570078 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:02.570936 17359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.570983 17357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
W20251028 21:10:02.571139 17360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.571332 17362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.571166 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:02.571486 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:02.571516 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:02.571532 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685802571533 us; error 0 us; skew 500 ppm
I20251028 21:10:02.572059 16180 webserver.cc:492] Webserver started at http://127.15.205.60:40721/ using document root <none> and password file <none>
I20251028 21:10:02.572160 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:02.572206 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:02.572247 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:02.572475 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root/instance:
uuid: "6963766ca50542be93c9757df33515b2"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.573745 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:02.574286 17368 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.574304 17357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:10:02.574421 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20251028 21:10:02.574467 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root
uuid: "6963766ca50542be93c9757df33515b2"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.574510 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:02.577023 17357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
W20251028 21:10:02.577564 17357 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.15.205.60:43203: Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111)
I20251028 21:10:02.584000 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:02.584151 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:02.586513 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.60:43203
I20251028 21:10:02.586539 17422 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.60:43203 every 8 connection(s)
I20251028 21:10:02.586680 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:10:02.586771 17423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:02.587342 17423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:10:02.589602 17423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:10:02.591717 17423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:10:02.594458 17423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2: Bootstrap starting.
I20251028 21:10:02.594730 17423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:02.595134 17423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2: No bootstrap required, opened a new log
I20251028 21:10:02.595244 17423 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.595322 17423 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:02.595345 17423 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6963766ca50542be93c9757df33515b2, State: Initialized, Role: FOLLOWER
I20251028 21:10:02.595393 17423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.595553 17423 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:10:02.595571 17428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.595642 17428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: This master's current role is: FOLLOWER
W20251028 21:10:02.596526 17439 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 21:10:02.596561 17439 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 21:10:02.606397 17291 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } attempt: 1
I20251028 21:10:02.608585 17291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } has no permanent_uuid. Determining permanent_uuid...
I20251028 21:10:02.611003 17291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: Bootstrap starting.
I20251028 21:10:02.611249 17291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:02.611717 17291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: No bootstrap required, opened a new log
I20251028 21:10:02.611807 17291 raft_consensus.cc:359] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.611850 17291 raft_consensus.cc:385] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:02.611863 17291 raft_consensus.cc:740] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 516e00ec9b3642e182a2b2bf08ff76b0, State: Initialized, Role: FOLLOWER
I20251028 21:10:02.611898 17291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.612042 17291 sys_catalog.cc:565] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:10:02.612071 17441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.612159 17441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:10:02.613088 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251028 21:10:02.613183 17452 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 21:10:02.613212 17452 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 21:10:02.623724 17357 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } attempt: 1
I20251028 21:10:02.626451 17357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae: Bootstrap starting.
I20251028 21:10:02.626792 17357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:02.627321 17357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae: No bootstrap required, opened a new log
I20251028 21:10:02.627456 17357 raft_consensus.cc:359] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.627501 17357 raft_consensus.cc:385] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:02.627524 17357 raft_consensus.cc:740] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc891714934a4ab78f8f7be7a9c799ae, State: Initialized, Role: FOLLOWER
I20251028 21:10:02.627575 17357 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.627719 17357 sys_catalog.cc:565] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:10:02.627725 17454 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.627808 17454 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:10:02.628706 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20251028 21:10:02.628803 17465 catalog_manager.cc:1568] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 21:10:02.628844 17465 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 21:10:02.628957 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:02.630070 17466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.630183 17469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.630203 16180 server_base.cc:1047] running on GCE node
W20251028 21:10:02.630213 17467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.630429 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:02.630462 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:02.630474 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685802630475 us; error 0 us; skew 500 ppm
I20251028 21:10:02.631018 16180 webserver.cc:492] Webserver started at http://127.15.205.1:37973/ using document root <none> and password file <none>
I20251028 21:10:02.631080 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:02.631110 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:02.631137 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:02.631379 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "bb1471646fa346c196227e0b6b493b00"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.632192 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:02.632630 17474 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.632757 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.632797 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "bb1471646fa346c196227e0b6b493b00"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.632819 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:02.641435 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:02.641638 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:02.641758 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:02.641933 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:02.641963 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.641989 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:02.642017 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.644995 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:37623
I20251028 21:10:02.645040 17537 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:37623 every 8 connection(s)
I20251028 21:10:02.645407 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:02.646936 17546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.647099 17545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.647815 17548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.648655 17540 heartbeater.cc:344] Connected to a master server at 127.15.205.61:41801
I20251028 21:10:02.648720 17540 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.648833 17540 heartbeater.cc:507] Master 127.15.205.61:41801 requested a full tablet report, sending...
I20251028 21:10:02.648885 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:02.648878 17538 heartbeater.cc:344] Connected to a master server at 127.15.205.62:36989
I20251028 21:10:02.648967 17538 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.649051 17538 heartbeater.cc:507] Master 127.15.205.62:36989 requested a full tablet report, sending...
I20251028 21:10:02.649080 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251028 21:10:02.649116 17321 ts_manager.cc:194] Registered new tserver with Master: bb1471646fa346c196227e0b6b493b00 (127.15.205.1:37623)
W20251028 21:10:02.649120 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:02.649191 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685802649191 us; error 0 us; skew 500 ppm
I20251028 21:10:02.649276 17255 ts_manager.cc:194] Registered new tserver with Master: bb1471646fa346c196227e0b6b493b00 (127.15.205.1:37623)
I20251028 21:10:02.649519 17539 heartbeater.cc:344] Connected to a master server at 127.15.205.60:43203
I20251028 21:10:02.649562 17539 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.649659 17539 heartbeater.cc:507] Master 127.15.205.60:43203 requested a full tablet report, sending...
I20251028 21:10:02.649746 16180 webserver.cc:492] Webserver started at http://127.15.205.2:33941/ using document root <none> and password file <none>
I20251028 21:10:02.649824 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:02.649875 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:02.649896 17387 ts_manager.cc:194] Registered new tserver with Master: bb1471646fa346c196227e0b6b493b00 (127.15.205.1:37623)
I20251028 21:10:02.649922 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:02.650130 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "2f2907483bd2441fbeb836e3ebd3efe2"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.650815 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:02.651175 17553 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.651326 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.651367 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "2f2907483bd2441fbeb836e3ebd3efe2"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.651409 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:02.655128 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:02.655483 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:02.655630 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:02.655807 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:02.655833 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.655859 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:02.655880 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.658869 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:46319
I20251028 21:10:02.658910 17616 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:46319 every 8 connection(s)
I20251028 21:10:02.659348 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:02.660749 17624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.660894 17627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:02.661371 17625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:02.662679 17618 heartbeater.cc:344] Connected to a master server at 127.15.205.60:43203
I20251028 21:10:02.662705 17617 heartbeater.cc:344] Connected to a master server at 127.15.205.62:36989
I20251028 21:10:02.662761 17618 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.662783 17617 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.662859 17618 heartbeater.cc:507] Master 127.15.205.60:43203 requested a full tablet report, sending...
I20251028 21:10:02.662917 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:02.662878 17619 heartbeater.cc:344] Connected to a master server at 127.15.205.61:41801
I20251028 21:10:02.663012 17619 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.663024 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:02.663048 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:02.663074 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685802663074 us; error 0 us; skew 500 ppm
I20251028 21:10:02.663105 17619 heartbeater.cc:507] Master 127.15.205.61:41801 requested a full tablet report, sending...
I20251028 21:10:02.663134 17387 ts_manager.cc:194] Registered new tserver with Master: 2f2907483bd2441fbeb836e3ebd3efe2 (127.15.205.2:46319)
I20251028 21:10:02.662881 17617 heartbeater.cc:507] Master 127.15.205.62:36989 requested a full tablet report, sending...
I20251028 21:10:02.663362 17255 ts_manager.cc:194] Registered new tserver with Master: 2f2907483bd2441fbeb836e3ebd3efe2 (127.15.205.2:46319)
I20251028 21:10:02.663417 17321 ts_manager.cc:194] Registered new tserver with Master: 2f2907483bd2441fbeb836e3ebd3efe2 (127.15.205.2:46319)
I20251028 21:10:02.663671 16180 webserver.cc:492] Webserver started at http://127.15.205.3:38867/ using document root <none> and password file <none>
I20251028 21:10:02.663738 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:02.663779 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:02.663821 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:02.664022 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "6209b089b57d4a9da52563e3d1d5b0f8"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.664736 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:02.665218 17632 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.665336 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.665377 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "6209b089b57d4a9da52563e3d1d5b0f8"
format_stamp: "Formatted at 2025-10-28 21:10:02 on dist-test-slave-zmkx"
I20251028 21:10:02.665421 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:02.682751 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:02.682940 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:02.683061 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:02.683231 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:02.683256 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.683315 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:02.683341 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:02.686235 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:42955
I20251028 21:10:02.686296 17695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:42955 every 8 connection(s)
I20251028 21:10:02.689214 17696 heartbeater.cc:344] Connected to a master server at 127.15.205.62:36989
I20251028 21:10:02.689271 17696 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.689365 17696 heartbeater.cc:507] Master 127.15.205.62:36989 requested a full tablet report, sending...
I20251028 21:10:02.689503 17698 heartbeater.cc:344] Connected to a master server at 127.15.205.61:41801
I20251028 21:10:02.689558 17698 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.689615 17255 ts_manager.cc:194] Registered new tserver with Master: 6209b089b57d4a9da52563e3d1d5b0f8 (127.15.205.3:42955)
I20251028 21:10:02.689637 17698 heartbeater.cc:507] Master 127.15.205.61:41801 requested a full tablet report, sending...
I20251028 21:10:02.689816 17697 heartbeater.cc:344] Connected to a master server at 127.15.205.60:43203
I20251028 21:10:02.689846 17321 ts_manager.cc:194] Registered new tserver with Master: 6209b089b57d4a9da52563e3d1d5b0f8 (127.15.205.3:42955)
I20251028 21:10:02.689863 17697 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:02.690049 17697 heartbeater.cc:507] Master 127.15.205.60:43203 requested a full tablet report, sending...
I20251028 21:10:02.690254 17387 ts_manager.cc:194] Registered new tserver with Master: 6209b089b57d4a9da52563e3d1d5b0f8 (127.15.205.3:42955)
I20251028 21:10:02.691205 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004713488s
I20251028 21:10:02.790028 17428 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:10:02.790143 17428 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.790287 17428 leader_election.cc:290] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 516e00ec9b3642e182a2b2bf08ff76b0 (127.15.205.62:36989), fc891714934a4ab78f8f7be7a9c799ae (127.15.205.61:41801)
I20251028 21:10:02.790517 17331 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6963766ca50542be93c9757df33515b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc891714934a4ab78f8f7be7a9c799ae" is_pre_election: true
I20251028 21:10:02.790517 17265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6963766ca50542be93c9757df33515b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" is_pre_election: true
I20251028 21:10:02.790620 17331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6963766ca50542be93c9757df33515b2 in term 0.
I20251028 21:10:02.790624 17265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6963766ca50542be93c9757df33515b2 in term 0.
I20251028 21:10:02.790776 17371 leader_election.cc:304] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6963766ca50542be93c9757df33515b2, fc891714934a4ab78f8f7be7a9c799ae; no voters:
I20251028 21:10:02.790877 17428 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:10:02.790956 17428 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:10:02.790985 17428 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:02.791693 17428 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.791827 17428 leader_election.cc:290] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [CANDIDATE]: Term 1 election: Requested vote from peers 516e00ec9b3642e182a2b2bf08ff76b0 (127.15.205.62:36989), fc891714934a4ab78f8f7be7a9c799ae (127.15.205.61:41801)
I20251028 21:10:02.791960 17331 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6963766ca50542be93c9757df33515b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc891714934a4ab78f8f7be7a9c799ae"
I20251028 21:10:02.791960 17265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6963766ca50542be93c9757df33515b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "516e00ec9b3642e182a2b2bf08ff76b0"
I20251028 21:10:02.792050 17331 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:02.792059 17265 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:02.792752 17265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6963766ca50542be93c9757df33515b2 in term 1.
I20251028 21:10:02.792752 17331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6963766ca50542be93c9757df33515b2 in term 1.
I20251028 21:10:02.792944 17372 leader_election.cc:304] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 516e00ec9b3642e182a2b2bf08ff76b0, 6963766ca50542be93c9757df33515b2; no voters:
I20251028 21:10:02.793027 17428 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:02.793148 17428 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 1 LEADER]: Becoming Leader. State: Replica: 6963766ca50542be93c9757df33515b2, State: Running, Role: LEADER
I20251028 21:10:02.793188 17428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:02.793462 17704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6963766ca50542be93c9757df33515b2. Latest consensus state: current_term: 1 leader_uuid: "6963766ca50542be93c9757df33515b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.793555 17704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:02.793704 17706 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:10:02.793862 17706 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:10:02.794363 17265 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 1 FOLLOWER]: Refusing update from remote peer 6963766ca50542be93c9757df33515b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:02.794399 17331 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 1 FOLLOWER]: Refusing update from remote peer 6963766ca50542be93c9757df33515b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:02.794517 17704 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:10:02.794587 17428 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:10:02.795070 17454 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6963766ca50542be93c9757df33515b2. Latest consensus state: current_term: 1 leader_uuid: "6963766ca50542be93c9757df33515b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.795147 17454 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:10:02.795135 17441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6963766ca50542be93c9757df33515b2. Latest consensus state: current_term: 1 leader_uuid: "6963766ca50542be93c9757df33515b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.795221 17441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:10:02.795492 17428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6963766ca50542be93c9757df33515b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.795565 17428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:02.795583 17706 catalog_manager.cc:1357] Generated new cluster ID: 20683ede7cd647269742717551026182
I20251028 21:10:02.795619 17706 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:10:02.795655 17428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6963766ca50542be93c9757df33515b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.795706 17428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:02.795923 17441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6963766ca50542be93c9757df33515b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.796000 17441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:10:02.796064 17454 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6963766ca50542be93c9757df33515b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:02.796110 17454 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:10:02.820013 17706 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:10:02.820458 17706 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:10:02.824919 17706 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2: Generated new TSK 0
I20251028 21:10:02.824996 17706 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:10:03.613425 17452 catalog_manager.cc:1269] Loaded cluster ID: 20683ede7cd647269742717551026182
I20251028 21:10:03.613490 17452 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: loading cluster ID for follower catalog manager: success
I20251028 21:10:03.614069 17452 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: acquiring CA information for follower catalog manager: success
I20251028 21:10:03.614158 17452 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 21:10:03.629074 17465 catalog_manager.cc:1269] Loaded cluster ID: 20683ede7cd647269742717551026182
I20251028 21:10:03.629138 17465 catalog_manager.cc:1562] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae: loading cluster ID for follower catalog manager: success
I20251028 21:10:03.629997 17465 catalog_manager.cc:1584] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae: acquiring CA information for follower catalog manager: success
I20251028 21:10:03.630139 17465 catalog_manager.cc:1612] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 21:10:03.651176 17387 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34664
I20251028 21:10:03.664084 17387 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34666
I20251028 21:10:03.690919 17387 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34672
I20251028 21:10:04.596505 17438 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:04.651763 17539 heartbeater.cc:499] Master 127.15.205.60:43203 was elected leader, sending a full tablet report...
I20251028 21:10:04.664494 17618 heartbeater.cc:499] Master 127.15.205.60:43203 was elected leader, sending a full tablet report...
I20251028 21:10:04.691342 17697 heartbeater.cc:499] Master 127.15.205.60:43203 was elected leader, sending a full tablet report...
I20251028 21:10:04.941928 16180 test_util.cc:276] Using random seed: 1889272421
I20251028 21:10:04.947077 17387 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49110:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 21:10:04.952822 17660 tablet_service.cc:1505] Processing CreateTablet for tablet 84681d9d7a3b45bcb5ae9076794a0aa9 (DEFAULT_TABLE table=test-workload [id=acece89775d541c0a9f0fc920bfc2e41]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251028 21:10:04.952983 17660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84681d9d7a3b45bcb5ae9076794a0aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:04.953358 17581 tablet_service.cc:1505] Processing CreateTablet for tablet c8850622d1104576ab38e2a5b72458df (DEFAULT_TABLE table=test-workload [id=acece89775d541c0a9f0fc920bfc2e41]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251028 21:10:04.953465 17581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8850622d1104576ab38e2a5b72458df. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:04.954326 17744 tablet_bootstrap.cc:492] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8: Bootstrap starting.
I20251028 21:10:04.954376 17502 tablet_service.cc:1505] Processing CreateTablet for tablet a9c1c02cad514a56a4c4d1811227a86c (DEFAULT_TABLE table=test-workload [id=acece89775d541c0a9f0fc920bfc2e41]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251028 21:10:04.954456 17502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9c1c02cad514a56a4c4d1811227a86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:04.954686 17744 tablet_bootstrap.cc:654] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:04.954968 17745 tablet_bootstrap.cc:492] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2: Bootstrap starting.
I20251028 21:10:04.955155 17744 tablet_bootstrap.cc:492] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8: No bootstrap required, opened a new log
I20251028 21:10:04.955186 17744 ts_tablet_manager.cc:1403] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:04.955344 17744 raft_consensus.cc:359] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209b089b57d4a9da52563e3d1d5b0f8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42955 } }
I20251028 21:10:04.955385 17744 raft_consensus.cc:385] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:04.955402 17744 raft_consensus.cc:740] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6209b089b57d4a9da52563e3d1d5b0f8, State: Initialized, Role: FOLLOWER
I20251028 21:10:04.955453 17744 consensus_queue.cc:260] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209b089b57d4a9da52563e3d1d5b0f8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42955 } }
I20251028 21:10:04.955483 17744 raft_consensus.cc:399] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:10:04.955504 17744 raft_consensus.cc:493] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:10:04.955525 17744 raft_consensus.cc:3060] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:04.955677 17747 tablet_bootstrap.cc:492] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00: Bootstrap starting.
I20251028 21:10:04.955999 17747 tablet_bootstrap.cc:654] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:04.956035 17744 raft_consensus.cc:515] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209b089b57d4a9da52563e3d1d5b0f8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42955 } }
I20251028 21:10:04.956104 17744 leader_election.cc:304] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6209b089b57d4a9da52563e3d1d5b0f8; no voters:
I20251028 21:10:04.956241 17744 leader_election.cc:290] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:10:04.956293 17749 raft_consensus.cc:2804] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:04.956367 17744 ts_tablet_manager.cc:1434] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:04.956390 17749 raft_consensus.cc:697] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 6209b089b57d4a9da52563e3d1d5b0f8, State: Running, Role: LEADER
I20251028 21:10:04.956660 17749 consensus_queue.cc:237] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209b089b57d4a9da52563e3d1d5b0f8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42955 } }
I20251028 21:10:04.956736 17747 tablet_bootstrap.cc:492] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00: No bootstrap required, opened a new log
I20251028 21:10:04.956781 17747 ts_tablet_manager.cc:1403] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:04.956997 17745 tablet_bootstrap.cc:654] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:04.957182 17387 catalog_manager.cc:5649] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6209b089b57d4a9da52563e3d1d5b0f8 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "6209b089b57d4a9da52563e3d1d5b0f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209b089b57d4a9da52563e3d1d5b0f8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42955 } health_report { overall_health: HEALTHY } } }
I20251028 21:10:04.957618 17747 raft_consensus.cc:359] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1471646fa346c196227e0b6b493b00" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 37623 } }
I20251028 21:10:04.957672 17747 raft_consensus.cc:385] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:04.957695 17747 raft_consensus.cc:740] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1471646fa346c196227e0b6b493b00, State: Initialized, Role: FOLLOWER
I20251028 21:10:04.957744 17747 consensus_queue.cc:260] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1471646fa346c196227e0b6b493b00" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 37623 } }
I20251028 21:10:04.957780 17747 raft_consensus.cc:399] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:10:04.957798 17747 raft_consensus.cc:493] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:10:04.957813 17747 raft_consensus.cc:3060] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:04.958320 17745 tablet_bootstrap.cc:492] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2: No bootstrap required, opened a new log
I20251028 21:10:04.958441 17747 raft_consensus.cc:515] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1471646fa346c196227e0b6b493b00" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 37623 } }
I20251028 21:10:04.958448 17745 ts_tablet_manager.cc:1403] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:10:04.958513 17747 leader_election.cc:304] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1471646fa346c196227e0b6b493b00; no voters:
I20251028 21:10:04.958611 17747 leader_election.cc:290] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:10:04.958647 17745 raft_consensus.cc:359] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2907483bd2441fbeb836e3ebd3efe2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 46319 } }
I20251028 21:10:04.958701 17745 raft_consensus.cc:385] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:04.958753 17745 raft_consensus.cc:740] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2907483bd2441fbeb836e3ebd3efe2, State: Initialized, Role: FOLLOWER
I20251028 21:10:04.958806 17755 raft_consensus.cc:2804] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:04.958858 17745 consensus_queue.cc:260] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2907483bd2441fbeb836e3ebd3efe2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 46319 } }
I20251028 21:10:04.958906 17745 raft_consensus.cc:399] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:10:04.958933 17745 raft_consensus.cc:493] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:10:04.958742 17747 ts_tablet_manager.cc:1434] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:10:04.958961 17745 raft_consensus.cc:3060] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:04.959564 17745 raft_consensus.cc:515] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2907483bd2441fbeb836e3ebd3efe2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 46319 } }
I20251028 21:10:04.959627 17745 leader_election.cc:304] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f2907483bd2441fbeb836e3ebd3efe2; no voters:
I20251028 21:10:04.959710 17745 leader_election.cc:290] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:10:04.959751 17758 raft_consensus.cc:2804] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:04.959827 17745 ts_tablet_manager.cc:1434] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:04.959949 17758 raft_consensus.cc:697] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 1 LEADER]: Becoming Leader. State: Replica: 2f2907483bd2441fbeb836e3ebd3efe2, State: Running, Role: LEADER
I20251028 21:10:04.960019 17758 consensus_queue.cc:237] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2907483bd2441fbeb836e3ebd3efe2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 46319 } }
I20251028 21:10:04.958948 17755 raft_consensus.cc:697] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 1 LEADER]: Becoming Leader. State: Replica: bb1471646fa346c196227e0b6b493b00, State: Running, Role: LEADER
I20251028 21:10:04.960233 17755 consensus_queue.cc:237] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1471646fa346c196227e0b6b493b00" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 37623 } }
I20251028 21:10:04.960388 17387 catalog_manager.cc:5649] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f2907483bd2441fbeb836e3ebd3efe2 (127.15.205.2). New cstate: current_term: 1 leader_uuid: "2f2907483bd2441fbeb836e3ebd3efe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2907483bd2441fbeb836e3ebd3efe2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 46319 } health_report { overall_health: HEALTHY } } }
I20251028 21:10:04.961452 17386 catalog_manager.cc:5649] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb1471646fa346c196227e0b6b493b00 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "bb1471646fa346c196227e0b6b493b00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1471646fa346c196227e0b6b493b00" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 37623 } health_report { overall_health: HEALTHY } } }
I20251028 21:10:04.962849 16180 master.cc:561] Master@127.15.205.60:43203 shutting down...
W20251028 21:10:04.963574 17539 heartbeater.cc:646] Failed to heartbeat to 127.15.205.60:43203 (0 consecutive failures): Remote error: Failed to send heartbeat to master: Service unavailable: service kudu.master.MasterService not registered on Master
I20251028 21:10:04.964881 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:04.964938 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:04.964972 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6963766ca50542be93c9757df33515b2: stopping tablet replica
I20251028 21:10:04.976940 16180 master.cc:583] Master@127.15.205.60:43203 shutdown complete.
I20251028 21:10:06.462846 17768 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6963766ca50542be93c9757df33515b2)
I20251028 21:10:06.462939 17769 raft_consensus.cc:493] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6963766ca50542be93c9757df33515b2)
I20251028 21:10:06.462939 17768 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:06.463068 17769 raft_consensus.cc:515] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:06.463183 17768 leader_election.cc:290] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 516e00ec9b3642e182a2b2bf08ff76b0 (127.15.205.62:36989), 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203)
I20251028 21:10:06.463395 17331 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "fc891714934a4ab78f8f7be7a9c799ae" is_pre_election: true
I20251028 21:10:06.463503 17331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 516e00ec9b3642e182a2b2bf08ff76b0 in term 1.
I20251028 21:10:06.463394 17769 leader_election.cc:290] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc891714934a4ab78f8f7be7a9c799ae (127.15.205.61:41801), 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203)
I20251028 21:10:06.463415 17265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc891714934a4ab78f8f7be7a9c799ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" is_pre_election: true
I20251028 21:10:06.463611 17265 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc891714934a4ab78f8f7be7a9c799ae in term 1.
I20251028 21:10:06.463763 17306 leader_election.cc:304] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 516e00ec9b3642e182a2b2bf08ff76b0, fc891714934a4ab78f8f7be7a9c799ae; no voters:
I20251028 21:10:06.463768 17240 leader_election.cc:304] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 516e00ec9b3642e182a2b2bf08ff76b0, fc891714934a4ab78f8f7be7a9c799ae; no voters:
I20251028 21:10:06.463958 17768 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 1 FOLLOWER]: Leader pre-election won for term 2
W20251028 21:10:06.463944 17305 leader_election.cc:336] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203): Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111)
I20251028 21:10:06.464016 17768 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6963766ca50542be93c9757df33515b2)
W20251028 21:10:06.464031 17240 leader_election.cc:336] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203): Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111)
I20251028 21:10:06.463953 17769 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 21:10:06.464038 17768 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:10:06.464123 17769 raft_consensus.cc:493] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6963766ca50542be93c9757df33515b2)
I20251028 21:10:06.464165 17769 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:10:06.464798 17768 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:06.464788 17769 raft_consensus.cc:515] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:06.464934 17768 leader_election.cc:290] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [CANDIDATE]: Term 2 election: Requested vote from peers 516e00ec9b3642e182a2b2bf08ff76b0 (127.15.205.62:36989), 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203)
I20251028 21:10:06.465060 17265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc891714934a4ab78f8f7be7a9c799ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "516e00ec9b3642e182a2b2bf08ff76b0"
I20251028 21:10:06.465060 17331 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "fc891714934a4ab78f8f7be7a9c799ae"
I20251028 21:10:06.465109 17769 leader_election.cc:290] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 2 election: Requested vote from peers fc891714934a4ab78f8f7be7a9c799ae (127.15.205.61:41801), 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203)
I20251028 21:10:06.465154 17265 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fc891714934a4ab78f8f7be7a9c799ae in current term 2: Already voted for candidate 516e00ec9b3642e182a2b2bf08ff76b0 in this term.
I20251028 21:10:06.465200 17331 raft_consensus.cc:2393] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 516e00ec9b3642e182a2b2bf08ff76b0 in current term 2: Already voted for candidate fc891714934a4ab78f8f7be7a9c799ae in this term.
W20251028 21:10:06.465495 17240 leader_election.cc:336] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203): Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111)
W20251028 21:10:06.465531 17305 leader_election.cc:336] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203): Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111)
I20251028 21:10:06.465575 17240 leader_election.cc:304] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 516e00ec9b3642e182a2b2bf08ff76b0; no voters: 6963766ca50542be93c9757df33515b2, fc891714934a4ab78f8f7be7a9c799ae
I20251028 21:10:06.465583 17305 leader_election.cc:304] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc891714934a4ab78f8f7be7a9c799ae; no voters: 516e00ec9b3642e182a2b2bf08ff76b0, 6963766ca50542be93c9757df33515b2
I20251028 21:10:06.465659 17769 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251028 21:10:06.465673 17768 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251028 21:10:08.060375 17780 raft_consensus.cc:493] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:10:08.060470 17780 raft_consensus.cc:515] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:08.060633 17780 leader_election.cc:290] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fc891714934a4ab78f8f7be7a9c799ae (127.15.205.61:41801), 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203)
I20251028 21:10:08.060837 17331 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "fc891714934a4ab78f8f7be7a9c799ae" is_pre_election: true
I20251028 21:10:08.060940 17331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 516e00ec9b3642e182a2b2bf08ff76b0 in term 2.
W20251028 21:10:08.061043 17240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20251028 21:10:08.061221 17240 leader_election.cc:304] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 516e00ec9b3642e182a2b2bf08ff76b0, fc891714934a4ab78f8f7be7a9c799ae; no voters:
I20251028 21:10:08.061300 17780 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251028 21:10:08.061347 17780 raft_consensus.cc:493] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:10:08.061369 17780 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 2 FOLLOWER]: Advancing to term 3
W20251028 21:10:08.061483 17240 leader_election.cc:336] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203): Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111)
I20251028 21:10:08.062088 17780 raft_consensus.cc:515] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:08.062193 17780 leader_election.cc:290] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 3 election: Requested vote from peers fc891714934a4ab78f8f7be7a9c799ae (127.15.205.61:41801), 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203)
I20251028 21:10:08.062345 17331 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "fc891714934a4ab78f8f7be7a9c799ae"
I20251028 21:10:08.062405 17331 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 2 FOLLOWER]: Advancing to term 3
W20251028 21:10:08.062655 17240 leader_election.cc:336] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203): Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111)
I20251028 21:10:08.062995 17331 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 516e00ec9b3642e182a2b2bf08ff76b0 in term 3.
I20251028 21:10:08.063151 17240 leader_election.cc:304] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 516e00ec9b3642e182a2b2bf08ff76b0, fc891714934a4ab78f8f7be7a9c799ae; no voters: 6963766ca50542be93c9757df33515b2
I20251028 21:10:08.063251 17780 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:10:08.063398 17780 raft_consensus.cc:697] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 3 LEADER]: Becoming Leader. State: Replica: 516e00ec9b3642e182a2b2bf08ff76b0, State: Running, Role: LEADER
I20251028 21:10:08.063460 17780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } }
I20251028 21:10:08.063714 17783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 516e00ec9b3642e182a2b2bf08ff76b0. Latest consensus state: current_term: 3 leader_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:08.063805 17783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:08.063952 17785 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:10:08.064159 17785 catalog_manager.cc:679] Loaded metadata for table test-workload [id=acece89775d541c0a9f0fc920bfc2e41]
I20251028 21:10:08.064276 17785 tablet_loader.cc:96] loaded metadata for tablet 84681d9d7a3b45bcb5ae9076794a0aa9 (table test-workload [id=acece89775d541c0a9f0fc920bfc2e41])
I20251028 21:10:08.064323 17785 tablet_loader.cc:96] loaded metadata for tablet a9c1c02cad514a56a4c4d1811227a86c (table test-workload [id=acece89775d541c0a9f0fc920bfc2e41])
I20251028 21:10:08.064365 17785 tablet_loader.cc:96] loaded metadata for tablet c8850622d1104576ab38e2a5b72458df (table test-workload [id=acece89775d541c0a9f0fc920bfc2e41])
I20251028 21:10:08.064395 17785 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:10:08.064469 17785 catalog_manager.cc:1269] Loaded cluster ID: 20683ede7cd647269742717551026182
I20251028 21:10:08.064507 17785 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:10:08.064668 17785 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:10:08.064736 17785 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: Loaded TSK: 0
I20251028 21:10:08.064838 17785 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:10:08.446647 17331 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 3 FOLLOWER]: Refusing update from remote peer 516e00ec9b3642e182a2b2bf08ff76b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20251028 21:10:08.446902 17783 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20251028 21:10:08.447366 17786 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 516e00ec9b3642e182a2b2bf08ff76b0. Latest consensus state: current_term: 3 leader_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:08.447466 17786 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: This master's current role is: FOLLOWER
I20251028 21:10:08.447894 17780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:08.448045 17780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:08.448055 17786 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "516e00ec9b3642e182a2b2bf08ff76b0" member_type: VOTER last_known_addr { host: "127.15.205.62" port: 36989 } } peers { permanent_uuid: "fc891714934a4ab78f8f7be7a9c799ae" member_type: VOTER last_known_addr { host: "127.15.205.61" port: 41801 } } peers { permanent_uuid: "6963766ca50542be93c9757df33515b2" member_type: VOTER last_known_addr { host: "127.15.205.60" port: 43203 } } }
I20251028 21:10:08.448179 17786 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [sys.catalog]: This master's current role is: FOLLOWER
W20251028 21:10:08.448433 17240 consensus_peers.cc:597] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 -> Peer 6963766ca50542be93c9757df33515b2 (127.15.205.60:43203): Couldn't send request to peer 6963766ca50542be93c9757df33515b2. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.60:43203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:10:08.613336 17451 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:08.613427 17451 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:09.109254 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:10:09.111629 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:09.111734 16180 tablet_replica.cc:333] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00: stopping tablet replica
I20251028 21:10:09.111778 16180 raft_consensus.cc:2243] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:09.111829 16180 raft_consensus.cc:2272] T a9c1c02cad514a56a4c4d1811227a86c P bb1471646fa346c196227e0b6b493b00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:09.123345 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:10:09.124119 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:10:09.126492 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:09.126576 16180 tablet_replica.cc:333] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2: stopping tablet replica
I20251028 21:10:09.126622 16180 raft_consensus.cc:2243] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:09.126665 16180 raft_consensus.cc:2272] T c8850622d1104576ab38e2a5b72458df P 2f2907483bd2441fbeb836e3ebd3efe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:09.137925 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:10:09.138523 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:10:09.141407 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:09.141520 16180 tablet_replica.cc:333] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8: stopping tablet replica
I20251028 21:10:09.141566 16180 raft_consensus.cc:2243] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:09.141605 16180 raft_consensus.cc:2272] T 84681d9d7a3b45bcb5ae9076794a0aa9 P 6209b089b57d4a9da52563e3d1d5b0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:09.152906 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:10:09.153594 16180 master.cc:561] Master@127.15.205.62:36989 shutting down...
I20251028 21:10:09.155681 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 3 LEADER]: Raft consensus shutting down.
I20251028 21:10:09.155750 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:09.155781 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 516e00ec9b3642e182a2b2bf08ff76b0: stopping tablet replica
I20251028 21:10:09.167249 16180 master.cc:583] Master@127.15.205.62:36989 shutdown complete.
I20251028 21:10:09.168285 16180 master.cc:561] Master@127.15.205.61:41801 shutting down...
I20251028 21:10:09.170435 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:09.170490 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:09.170507 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P fc891714934a4ab78f8f7be7a9c799ae: stopping tablet replica
I20251028 21:10:09.181977 16180 master.cc:583] Master@127.15.205.61:41801 shutdown complete.
[ OK ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing (6661 ms)
[ RUN ] AutoRebalancerTest.MovesScheduledIfAddTserver
I20251028 21:10:09.186523 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:45519
I20251028 21:10:09.186654 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:09.187808 17795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:09.187808 17797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:09.187877 17794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:09.188057 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:09.188145 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:09.188174 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:09.188199 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685809188199 us; error 0 us; skew 500 ppm
I20251028 21:10:09.188695 16180 webserver.cc:492] Webserver started at http://127.15.205.62:33879/ using document root <none> and password file <none>
I20251028 21:10:09.188771 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:09.188817 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:09.188869 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:09.189091 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "3fb90eb36603473ebe36c7dd73478537"
format_stamp: "Formatted at 2025-10-28 21:10:09 on dist-test-slave-zmkx"
I20251028 21:10:09.189826 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:09.190236 17802 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.190362 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.190404 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "3fb90eb36603473ebe36c7dd73478537"
format_stamp: "Formatted at 2025-10-28 21:10:09 on dist-test-slave-zmkx"
I20251028 21:10:09.190445 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:09.200908 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:09.201030 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:09.203248 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:45519
I20251028 21:10:09.205946 17854 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:45519 every 8 connection(s)
I20251028 21:10:09.206038 17855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:09.206812 17855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537: Bootstrap starting.
I20251028 21:10:09.207016 17855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:09.207477 17855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537: No bootstrap required, opened a new log
I20251028 21:10:09.207581 17855 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb90eb36603473ebe36c7dd73478537" member_type: VOTER }
I20251028 21:10:09.207619 17855 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:09.207633 17855 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb90eb36603473ebe36c7dd73478537, State: Initialized, Role: FOLLOWER
I20251028 21:10:09.207664 17855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb90eb36603473ebe36c7dd73478537" member_type: VOTER }
I20251028 21:10:09.207706 17855 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:10:09.207736 17855 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:10:09.207755 17855 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:09.208209 17855 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb90eb36603473ebe36c7dd73478537" member_type: VOTER }
I20251028 21:10:09.208258 17855 leader_election.cc:304] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fb90eb36603473ebe36c7dd73478537; no voters:
I20251028 21:10:09.208338 17855 leader_election.cc:290] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:10:09.208400 17858 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:09.208443 17855 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:10:09.208523 17858 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb90eb36603473ebe36c7dd73478537, State: Running, Role: LEADER
I20251028 21:10:09.208570 17858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb90eb36603473ebe36c7dd73478537" member_type: VOTER }
I20251028 21:10:09.208787 17859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fb90eb36603473ebe36c7dd73478537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb90eb36603473ebe36c7dd73478537" member_type: VOTER } }
I20251028 21:10:09.208896 17859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:09.208809 17860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fb90eb36603473ebe36c7dd73478537. Latest consensus state: current_term: 1 leader_uuid: "3fb90eb36603473ebe36c7dd73478537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb90eb36603473ebe36c7dd73478537" member_type: VOTER } }
I20251028 21:10:09.209034 17860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:09.209172 17864 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:10:09.209292 17864 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:10:09.209751 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:10:09.209758 17864 catalog_manager.cc:1357] Generated new cluster ID: 6603caa4a75040798a214cd2108a5c44
I20251028 21:10:09.209810 17864 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:10:09.229913 17864 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:10:09.230356 17864 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:10:09.233719 17864 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537: Generated new TSK 0
I20251028 21:10:09.233783 17864 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:10:09.241482 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:09.242528 17876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:09.242561 17877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:09.242594 17879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:09.242666 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:09.242777 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:09.242810 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:09.242831 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685809242832 us; error 0 us; skew 500 ppm
I20251028 21:10:09.243333 16180 webserver.cc:492] Webserver started at http://127.15.205.1:35431/ using document root <none> and password file <none>
I20251028 21:10:09.243392 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:09.243418 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:09.243448 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:09.243614 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "5a5736535d4846cf88bc0c691730894b"
format_stamp: "Formatted at 2025-10-28 21:10:09 on dist-test-slave-zmkx"
I20251028 21:10:09.244305 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:10:09.244658 17884 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.244771 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.244815 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "5a5736535d4846cf88bc0c691730894b"
format_stamp: "Formatted at 2025-10-28 21:10:09 on dist-test-slave-zmkx"
I20251028 21:10:09.244846 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:09.251489 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:09.251621 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:09.251703 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:09.251822 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:09.251842 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.251864 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:09.251881 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.254248 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:33911
I20251028 21:10:09.254274 17947 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:33911 every 8 connection(s)
I20251028 21:10:09.254525 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:09.255432 17952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:09.255478 17953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:09.255514 17955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:09.255677 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:09.255798 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:09.255825 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:09.255847 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685809255848 us; error 0 us; skew 500 ppm
I20251028 21:10:09.256294 16180 webserver.cc:492] Webserver started at http://127.15.205.2:39817/ using document root <none> and password file <none>
I20251028 21:10:09.256372 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:09.256413 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:09.256453 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:09.256652 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "145ca98f4dbc4e30aa7330ab56807f19"
format_stamp: "Formatted at 2025-10-28 21:10:09 on dist-test-slave-zmkx"
I20251028 21:10:09.257546 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:09.257576 17948 heartbeater.cc:344] Connected to a master server at 127.15.205.62:45519
I20251028 21:10:09.257627 17948 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:09.257726 17948 heartbeater.cc:507] Master 127.15.205.62:45519 requested a full tablet report, sending...
I20251028 21:10:09.257917 17819 ts_manager.cc:194] Registered new tserver with Master: 5a5736535d4846cf88bc0c691730894b (127.15.205.1:33911)
I20251028 21:10:09.258034 17960 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.258147 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.258188 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "145ca98f4dbc4e30aa7330ab56807f19"
format_stamp: "Formatted at 2025-10-28 21:10:09 on dist-test-slave-zmkx"
I20251028 21:10:09.258224 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:09.258536 17819 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58894
I20251028 21:10:09.276834 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:09.276975 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:09.277070 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:09.277199 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:09.277221 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.277238 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:09.277253 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.279688 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:44827
I20251028 21:10:09.279716 18023 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:44827 every 8 connection(s)
I20251028 21:10:09.279974 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:09.280953 18027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:09.280980 18028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:09.281068 18030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:09.281168 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:09.281318 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:09.281349 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:09.281363 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685809281362 us; error 0 us; skew 500 ppm
I20251028 21:10:09.281843 16180 webserver.cc:492] Webserver started at http://127.15.205.3:42703/ using document root <none> and password file <none>
I20251028 21:10:09.281932 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:09.281983 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:09.282030 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:09.282292 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "8bed3b51c142463a93a29d0001d0a915"
format_stamp: "Formatted at 2025-10-28 21:10:09 on dist-test-slave-zmkx"
I20251028 21:10:09.282857 18024 heartbeater.cc:344] Connected to a master server at 127.15.205.62:45519
I20251028 21:10:09.282918 18024 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:09.283004 18024 heartbeater.cc:507] Master 127.15.205.62:45519 requested a full tablet report, sending...
I20251028 21:10:09.283221 17819 ts_manager.cc:194] Registered new tserver with Master: 145ca98f4dbc4e30aa7330ab56807f19 (127.15.205.2:44827)
I20251028 21:10:09.283360 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:09.283675 17819 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58898
I20251028 21:10:09.283800 18035 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.283898 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.283948 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "8bed3b51c142463a93a29d0001d0a915"
format_stamp: "Formatted at 2025-10-28 21:10:09 on dist-test-slave-zmkx"
I20251028 21:10:09.284001 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:09.296608 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:09.296756 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:09.296868 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:09.297014 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:09.297037 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.297063 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:09.297087 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:09.299715 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:36989
I20251028 21:10:09.299762 18098 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:36989 every 8 connection(s)
I20251028 21:10:09.302023 18099 heartbeater.cc:344] Connected to a master server at 127.15.205.62:45519
I20251028 21:10:09.302073 18099 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:09.302175 18099 heartbeater.cc:507] Master 127.15.205.62:45519 requested a full tablet report, sending...
I20251028 21:10:09.302359 17819 ts_manager.cc:194] Registered new tserver with Master: 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3:36989)
I20251028 21:10:09.302655 17819 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58914
I20251028 21:10:09.303150 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003280502s
I20251028 21:10:10.258946 17948 heartbeater.cc:499] Master 127.15.205.62:45519 was elected leader, sending a full tablet report...
I20251028 21:10:10.284181 18024 heartbeater.cc:499] Master 127.15.205.62:45519 was elected leader, sending a full tablet report...
I20251028 21:10:10.303054 18099 heartbeater.cc:499] Master 127.15.205.62:45519 was elected leader, sending a full tablet report...
I20251028 21:10:10.327886 16180 test_util.cc:276] Using random seed: 1894658381
I20251028 21:10:10.332244 17819 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58922:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:10:10.332396 17819 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:10:10.337091 18063 tablet_service.cc:1505] Processing CreateTablet for tablet 7c197220a11644109953ed7947ccd908 (DEFAULT_TABLE table=test-workload [id=050b82e745d14eb5ac7792e930ab229a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251028 21:10:10.337141 17911 tablet_service.cc:1505] Processing CreateTablet for tablet cfad4b23af4f4f08ba66c40649131e63 (DEFAULT_TABLE table=test-workload [id=050b82e745d14eb5ac7792e930ab229a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251028 21:10:10.337100 17912 tablet_service.cc:1505] Processing CreateTablet for tablet 7c197220a11644109953ed7947ccd908 (DEFAULT_TABLE table=test-workload [id=050b82e745d14eb5ac7792e930ab229a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251028 21:10:10.337235 17911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfad4b23af4f4f08ba66c40649131e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:10.337278 18062 tablet_service.cc:1505] Processing CreateTablet for tablet cfad4b23af4f4f08ba66c40649131e63 (DEFAULT_TABLE table=test-workload [id=050b82e745d14eb5ac7792e930ab229a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251028 21:10:10.337298 17912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c197220a11644109953ed7947ccd908. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:10.337235 18063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c197220a11644109953ed7947ccd908. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:10.337498 18062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfad4b23af4f4f08ba66c40649131e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:10.337829 17988 tablet_service.cc:1505] Processing CreateTablet for tablet 7c197220a11644109953ed7947ccd908 (DEFAULT_TABLE table=test-workload [id=050b82e745d14eb5ac7792e930ab229a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251028 21:10:10.337913 17988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c197220a11644109953ed7947ccd908. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:10.338907 18120 tablet_bootstrap.cc:492] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915: Bootstrap starting.
I20251028 21:10:10.338999 18119 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b: Bootstrap starting.
I20251028 21:10:10.339258 17987 tablet_service.cc:1505] Processing CreateTablet for tablet cfad4b23af4f4f08ba66c40649131e63 (DEFAULT_TABLE table=test-workload [id=050b82e745d14eb5ac7792e930ab229a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251028 21:10:10.339378 17987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfad4b23af4f4f08ba66c40649131e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:10.339423 18120 tablet_bootstrap.cc:654] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:10.339586 18119 tablet_bootstrap.cc:654] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:10.340098 18119 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b: No bootstrap required, opened a new log
I20251028 21:10:10.340139 18119 ts_tablet_manager.cc:1403] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:10.340266 18120 tablet_bootstrap.cc:492] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915: No bootstrap required, opened a new log
I20251028 21:10:10.340301 18120 ts_tablet_manager.cc:1403] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:10.340278 18119 raft_consensus.cc:359] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.340334 18119 raft_consensus.cc:385] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:10.340353 18119 raft_consensus.cc:740] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a5736535d4846cf88bc0c691730894b, State: Initialized, Role: FOLLOWER
I20251028 21:10:10.340404 18119 consensus_queue.cc:260] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.340426 18120 raft_consensus.cc:359] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.340476 18120 raft_consensus.cc:385] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:10.340488 18120 raft_consensus.cc:740] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bed3b51c142463a93a29d0001d0a915, State: Initialized, Role: FOLLOWER
I20251028 21:10:10.340529 18120 consensus_queue.cc:260] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.340545 18119 ts_tablet_manager.cc:1434] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:10:10.340605 18119 tablet_bootstrap.cc:492] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b: Bootstrap starting.
I20251028 21:10:10.340668 18120 ts_tablet_manager.cc:1434] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:10.340741 18120 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: Bootstrap starting.
I20251028 21:10:10.340997 18119 tablet_bootstrap.cc:654] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:10.341143 18121 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19: Bootstrap starting.
I20251028 21:10:10.341156 18120 tablet_bootstrap.cc:654] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:10.341568 18119 tablet_bootstrap.cc:492] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b: No bootstrap required, opened a new log
I20251028 21:10:10.341589 18121 tablet_bootstrap.cc:654] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:10.341634 18119 ts_tablet_manager.cc:1403] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:10.341678 18120 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: No bootstrap required, opened a new log
I20251028 21:10:10.341712 18120 ts_tablet_manager.cc:1403] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:10.341775 18119 raft_consensus.cc:359] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.341827 18120 raft_consensus.cc:359] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.341923 18120 raft_consensus.cc:385] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:10.341951 18120 raft_consensus.cc:740] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bed3b51c142463a93a29d0001d0a915, State: Initialized, Role: FOLLOWER
I20251028 21:10:10.342024 18120 consensus_queue.cc:260] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.342114 18120 ts_tablet_manager.cc:1434] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:10.342144 18121 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19: No bootstrap required, opened a new log
I20251028 21:10:10.341826 18119 raft_consensus.cc:385] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:10.342192 18121 ts_tablet_manager.cc:1403] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:10.342247 18119 raft_consensus.cc:740] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a5736535d4846cf88bc0c691730894b, State: Initialized, Role: FOLLOWER
I20251028 21:10:10.342303 18119 consensus_queue.cc:260] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.342381 18121 raft_consensus.cc:359] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.342384 18119 ts_tablet_manager.cc:1434] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:10:10.342458 18121 raft_consensus.cc:385] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:10.342480 18121 raft_consensus.cc:740] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145ca98f4dbc4e30aa7330ab56807f19, State: Initialized, Role: FOLLOWER
I20251028 21:10:10.342520 18121 consensus_queue.cc:260] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.342661 18121 ts_tablet_manager.cc:1434] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:10.342716 18121 tablet_bootstrap.cc:492] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19: Bootstrap starting.
I20251028 21:10:10.343084 18121 tablet_bootstrap.cc:654] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:10.343622 18121 tablet_bootstrap.cc:492] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19: No bootstrap required, opened a new log
I20251028 21:10:10.343667 18121 ts_tablet_manager.cc:1403] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:10:10.343808 18121 raft_consensus.cc:359] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.343865 18121 raft_consensus.cc:385] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:10.343886 18121 raft_consensus.cc:740] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145ca98f4dbc4e30aa7330ab56807f19, State: Initialized, Role: FOLLOWER
I20251028 21:10:10.343932 18121 consensus_queue.cc:260] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.344019 18121 ts_tablet_manager.cc:1434] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:10.389480 18124 raft_consensus.cc:493] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:10:10.389536 18124 raft_consensus.cc:515] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.389685 18124 leader_election.cc:290] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145ca98f4dbc4e30aa7330ab56807f19 (127.15.205.2:44827), 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3:36989)
I20251028 21:10:10.392812 18073 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c197220a11644109953ed7947ccd908" candidate_uuid: "5a5736535d4846cf88bc0c691730894b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bed3b51c142463a93a29d0001d0a915" is_pre_election: true
I20251028 21:10:10.392879 17998 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c197220a11644109953ed7947ccd908" candidate_uuid: "5a5736535d4846cf88bc0c691730894b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145ca98f4dbc4e30aa7330ab56807f19" is_pre_election: true
I20251028 21:10:10.392885 18073 raft_consensus.cc:2468] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a5736535d4846cf88bc0c691730894b in term 0.
I20251028 21:10:10.392963 17998 raft_consensus.cc:2468] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a5736535d4846cf88bc0c691730894b in term 0.
I20251028 21:10:10.393085 17887 leader_election.cc:304] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a5736535d4846cf88bc0c691730894b, 8bed3b51c142463a93a29d0001d0a915; no voters:
I20251028 21:10:10.393179 18124 raft_consensus.cc:2804] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:10:10.393208 18124 raft_consensus.cc:493] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:10:10.393221 18124 raft_consensus.cc:3060] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:10.393700 18124 raft_consensus.cc:515] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.393803 18124 leader_election.cc:290] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [CANDIDATE]: Term 1 election: Requested vote from peers 145ca98f4dbc4e30aa7330ab56807f19 (127.15.205.2:44827), 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3:36989)
I20251028 21:10:10.393949 18073 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c197220a11644109953ed7947ccd908" candidate_uuid: "5a5736535d4846cf88bc0c691730894b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bed3b51c142463a93a29d0001d0a915"
I20251028 21:10:10.393975 17998 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7c197220a11644109953ed7947ccd908" candidate_uuid: "5a5736535d4846cf88bc0c691730894b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145ca98f4dbc4e30aa7330ab56807f19"
I20251028 21:10:10.393998 18073 raft_consensus.cc:3060] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:10.394018 17998 raft_consensus.cc:3060] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:10.394528 18073 raft_consensus.cc:2468] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a5736535d4846cf88bc0c691730894b in term 1.
I20251028 21:10:10.394556 17998 raft_consensus.cc:2468] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a5736535d4846cf88bc0c691730894b in term 1.
I20251028 21:10:10.394691 17888 leader_election.cc:304] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145ca98f4dbc4e30aa7330ab56807f19, 5a5736535d4846cf88bc0c691730894b; no voters:
I20251028 21:10:10.394793 18124 raft_consensus.cc:2804] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:10.394901 18124 raft_consensus.cc:697] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 1 LEADER]: Becoming Leader. State: Replica: 5a5736535d4846cf88bc0c691730894b, State: Running, Role: LEADER
I20251028 21:10:10.394961 18124 consensus_queue.cc:237] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.395406 17818 catalog_manager.cc:5649] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a5736535d4846cf88bc0c691730894b (127.15.205.1). New cstate: current_term: 1 leader_uuid: "5a5736535d4846cf88bc0c691730894b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } health_report { overall_health: UNKNOWN } } }
I20251028 21:10:10.469947 18125 raft_consensus.cc:493] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:10:10.470006 18125 raft_consensus.cc:515] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.470155 18125 leader_election.cc:290] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a5736535d4846cf88bc0c691730894b (127.15.205.1:33911), 145ca98f4dbc4e30aa7330ab56807f19 (127.15.205.2:44827)
I20251028 21:10:10.473326 17998 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfad4b23af4f4f08ba66c40649131e63" candidate_uuid: "8bed3b51c142463a93a29d0001d0a915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145ca98f4dbc4e30aa7330ab56807f19" is_pre_election: true
I20251028 21:10:10.473347 17922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfad4b23af4f4f08ba66c40649131e63" candidate_uuid: "8bed3b51c142463a93a29d0001d0a915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a5736535d4846cf88bc0c691730894b" is_pre_election: true
I20251028 21:10:10.473438 17998 raft_consensus.cc:2468] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bed3b51c142463a93a29d0001d0a915 in term 0.
I20251028 21:10:10.473444 17922 raft_consensus.cc:2468] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bed3b51c142463a93a29d0001d0a915 in term 0.
I20251028 21:10:10.473587 18038 leader_election.cc:304] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a5736535d4846cf88bc0c691730894b, 8bed3b51c142463a93a29d0001d0a915; no voters:
I20251028 21:10:10.473686 18125 raft_consensus.cc:2804] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:10:10.473734 18125 raft_consensus.cc:493] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:10:10.473758 18125 raft_consensus.cc:3060] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:10.474263 18125 raft_consensus.cc:515] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.474370 18125 leader_election.cc:290] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [CANDIDATE]: Term 1 election: Requested vote from peers 5a5736535d4846cf88bc0c691730894b (127.15.205.1:33911), 145ca98f4dbc4e30aa7330ab56807f19 (127.15.205.2:44827)
I20251028 21:10:10.474512 17922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfad4b23af4f4f08ba66c40649131e63" candidate_uuid: "8bed3b51c142463a93a29d0001d0a915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a5736535d4846cf88bc0c691730894b"
I20251028 21:10:10.474529 17998 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfad4b23af4f4f08ba66c40649131e63" candidate_uuid: "8bed3b51c142463a93a29d0001d0a915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145ca98f4dbc4e30aa7330ab56807f19"
I20251028 21:10:10.474563 17922 raft_consensus.cc:3060] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:10.474568 17998 raft_consensus.cc:3060] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:10.475028 17998 raft_consensus.cc:2468] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bed3b51c142463a93a29d0001d0a915 in term 1.
I20251028 21:10:10.475112 17922 raft_consensus.cc:2468] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bed3b51c142463a93a29d0001d0a915 in term 1.
I20251028 21:10:10.475198 18039 leader_election.cc:304] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145ca98f4dbc4e30aa7330ab56807f19, 8bed3b51c142463a93a29d0001d0a915; no voters:
I20251028 21:10:10.475304 18125 raft_consensus.cc:2804] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:10.475423 18125 raft_consensus.cc:697] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 1 LEADER]: Becoming Leader. State: Replica: 8bed3b51c142463a93a29d0001d0a915, State: Running, Role: LEADER
I20251028 21:10:10.475471 18125 consensus_queue.cc:237] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } }
I20251028 21:10:10.475941 17818 catalog_manager.cc:5649] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "8bed3b51c142463a93a29d0001d0a915" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } health_report { overall_health: HEALTHY } } }
I20251028 21:10:10.941977 18139 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:10:10.943461 18143 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:10:10.980176 18125 consensus_queue.cc:1048] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:10:10.981149 18149 consensus_queue.cc:1048] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:10:11.209823 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:11.209954 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:11.210002 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:13.210167 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:13.210317 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:13.210361 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:15.210494 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:15.210630 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:15.210661 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:17.210791 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:17.210923 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:17.210975 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:19.211133 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:19.211293 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:19.211328 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:21.211462 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:21.211594 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:21.211627 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:21.528903 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:21.530154 18152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:21.530226 18155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:21.530392 16180 server_base.cc:1047] running on GCE node
W20251028 21:10:21.530421 18153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:21.530563 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:21.530593 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:21.530614 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685821530614 us; error 0 us; skew 500 ppm
I20251028 21:10:21.531232 16180 webserver.cc:492] Webserver started at http://127.15.205.4:35151/ using document root <none> and password file <none>
I20251028 21:10:21.531344 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:21.531396 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:21.531447 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:21.531715 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-3-root/instance:
uuid: "eb553c81bf674f0a94f810cb615e33c0"
format_stamp: "Formatted at 2025-10-28 21:10:21 on dist-test-slave-zmkx"
I20251028 21:10:21.532531 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:21.532987 18160 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:21.533097 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:21.533151 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-3-root
uuid: "eb553c81bf674f0a94f810cb615e33c0"
format_stamp: "Formatted at 2025-10-28 21:10:21 on dist-test-slave-zmkx"
I20251028 21:10:21.533192 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:21.541769 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:21.541924 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:21.542047 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:21.542229 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:21.542256 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:21.542274 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:21.542300 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:21.545358 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.4:34531
I20251028 21:10:21.545399 18223 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.4:34531 every 8 connection(s)
I20251028 21:10:21.553030 18224 heartbeater.cc:344] Connected to a master server at 127.15.205.62:45519
I20251028 21:10:21.553079 18224 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:21.553148 18224 heartbeater.cc:507] Master 127.15.205.62:45519 requested a full tablet report, sending...
I20251028 21:10:21.553498 17819 ts_manager.cc:194] Registered new tserver with Master: eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4:34531)
I20251028 21:10:21.553781 17819 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51110
I20251028 21:10:22.215238 17922 consensus_queue.cc:237] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: true } }
I20251028 21:10:22.215896 17998 raft_consensus.cc:1275] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Refusing update from remote peer 5a5736535d4846cf88bc0c691730894b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:22.215919 18073 raft_consensus.cc:1275] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Refusing update from remote peer 5a5736535d4846cf88bc0c691730894b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:10:22.216157 18232 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:22.216271 18231 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:10:22.216921 18232 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: true } } }
I20251028 21:10:22.217025 17998 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: true } } }
I20251028 21:10:22.217968 17819 catalog_manager.cc:5649] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b reported cstate change: config changed from index -1 to 2, NON_VOTER eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "5a5736535d4846cf88bc0c691730894b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:10:22.218065 18073 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: true } } }
W20251028 21:10:22.218685 17885 consensus_peers.cc:597] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b -> Peer eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4:34531): Couldn't send request to peer eb553c81bf674f0a94f810cb615e33c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c197220a11644109953ed7947ccd908. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:10:22.554199 18224 heartbeater.cc:499] Master 127.15.205.62:45519 was elected leader, sending a full tablet report...
I20251028 21:10:22.732005 18243 ts_tablet_manager.cc:933] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: Initiating tablet copy from peer 5a5736535d4846cf88bc0c691730894b (127.15.205.1:33911)
I20251028 21:10:22.732259 18243 tablet_copy_client.cc:323] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.1:33911
I20251028 21:10:22.735512 17932 tablet_copy_service.cc:140] P 5a5736535d4846cf88bc0c691730894b: Received BeginTabletCopySession request for tablet 7c197220a11644109953ed7947ccd908 from peer eb553c81bf674f0a94f810cb615e33c0 ({username='slave'} at 127.0.0.1:36810)
I20251028 21:10:22.735582 17932 tablet_copy_service.cc:161] P 5a5736535d4846cf88bc0c691730894b: Beginning new tablet copy session on tablet 7c197220a11644109953ed7947ccd908 from peer eb553c81bf674f0a94f810cb615e33c0 at {username='slave'} at 127.0.0.1:36810: session id = eb553c81bf674f0a94f810cb615e33c0-7c197220a11644109953ed7947ccd908
I20251028 21:10:22.736351 17932 tablet_copy_source_session.cc:215] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:10:22.736874 18243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c197220a11644109953ed7947ccd908. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:22.738214 18243 tablet_copy_client.cc:806] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: tablet copy: Starting download of 0 data blocks...
I20251028 21:10:22.738341 18243 tablet_copy_client.cc:670] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: tablet copy: Starting download of 1 WAL segments...
I20251028 21:10:22.738888 18243 tablet_copy_client.cc:538] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:10:22.739672 18243 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: Bootstrap starting.
I20251028 21:10:22.741142 18243 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:10:22.741323 18243 tablet_bootstrap.cc:492] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: Bootstrap complete.
I20251028 21:10:22.741413 18243 ts_tablet_manager.cc:1403] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:10:22.741575 18243 raft_consensus.cc:359] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: true } }
I20251028 21:10:22.741636 18243 raft_consensus.cc:740] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb553c81bf674f0a94f810cb615e33c0, State: Initialized, Role: LEARNER
I20251028 21:10:22.741668 18243 consensus_queue.cc:260] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: true } }
I20251028 21:10:22.741833 18243 ts_tablet_manager.cc:1434] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251028 21:10:22.742017 17932 tablet_copy_service.cc:342] P 5a5736535d4846cf88bc0c691730894b: Request end of tablet copy session eb553c81bf674f0a94f810cb615e33c0-7c197220a11644109953ed7947ccd908 received from {username='slave'} at 127.0.0.1:36810
I20251028 21:10:22.742059 17932 tablet_copy_service.cc:434] P 5a5736535d4846cf88bc0c691730894b: ending tablet copy session eb553c81bf674f0a94f810cb615e33c0-7c197220a11644109953ed7947ccd908 on tablet 7c197220a11644109953ed7947ccd908 with peer eb553c81bf674f0a94f810cb615e33c0
I20251028 21:10:23.211737 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:23.211855 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:23.211903 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:23.255076 18198 raft_consensus.cc:1217] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251028 21:10:23.879447 18249 raft_consensus.cc:1064] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b: attempting to promote NON_VOTER eb553c81bf674f0a94f810cb615e33c0 to VOTER
I20251028 21:10:23.879689 18249 consensus_queue.cc:237] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } }
I20251028 21:10:23.880126 18198 raft_consensus.cc:1275] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 LEARNER]: Refusing update from remote peer 5a5736535d4846cf88bc0c691730894b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:10:23.880206 17998 raft_consensus.cc:1275] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Refusing update from remote peer 5a5736535d4846cf88bc0c691730894b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:10:23.880232 18073 raft_consensus.cc:1275] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Refusing update from remote peer 5a5736535d4846cf88bc0c691730894b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:10:23.880421 18251 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:10:23.880522 18249 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:10:23.880640 18254 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:10:23.882277 18253 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.882381 18198 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.883387 17998 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.883527 17817 catalog_manager.cc:5649] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 reported cstate change: config changed from index 2 to 3, eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5a5736535d4846cf88bc0c691730894b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.884378 18073 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb553c81bf674f0a94f810cb615e33c0 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "8bed3b51c142463a93a29d0001d0a915" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36989 } attrs { replace: true } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.885104 17922 consensus_queue.cc:237] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } }
I20251028 21:10:23.885435 18198 raft_consensus.cc:1275] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 FOLLOWER]: Refusing update from remote peer 5a5736535d4846cf88bc0c691730894b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:10:23.885623 18254 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:10:23.885675 17998 raft_consensus.cc:1275] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Refusing update from remote peer 5a5736535d4846cf88bc0c691730894b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:10:23.885818 18253 consensus_queue.cc:1048] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [LEADER]: Connected to new peer: Peer: permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:10:23.886099 18251 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.886204 18198 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.886844 17805 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 7c197220a11644109953ed7947ccd908 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:10:23.887142 17818 catalog_manager.cc:5649] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 reported cstate change: config changed from index 3 to 4, VOTER 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3) evicted. New cstate: current_term: 1 leader_uuid: "5a5736535d4846cf88bc0c691730894b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.887362 17998 raft_consensus.cc:2955] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5a5736535d4846cf88bc0c691730894b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 33911 } } peers { permanent_uuid: "145ca98f4dbc4e30aa7330ab56807f19" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44827 } } peers { permanent_uuid: "eb553c81bf674f0a94f810cb615e33c0" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 34531 } attrs { promote: false } } }
I20251028 21:10:23.887776 18063 tablet_service.cc:1552] Processing DeleteTablet for tablet 7c197220a11644109953ed7947ccd908 with delete_type TABLET_DATA_TOMBSTONED (TS 8bed3b51c142463a93a29d0001d0a915 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47438
I20251028 21:10:23.887971 18264 tablet_replica.cc:333] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: stopping tablet replica
I20251028 21:10:23.888149 18264 raft_consensus.cc:2243] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:23.888204 18264 raft_consensus.cc:2272] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:23.888383 18264 ts_tablet_manager.cc:1916] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:10:23.889271 18264 ts_tablet_manager.cc:1929] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251028 21:10:23.889317 18264 log.cc:1199] T 7c197220a11644109953ed7947ccd908 P 8bed3b51c142463a93a29d0001d0a915: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/7c197220a11644109953ed7947ccd908
I20251028 21:10:23.889598 17805 catalog_manager.cc:4985] TS 8bed3b51c142463a93a29d0001d0a915 (127.15.205.3:36989): tablet 7c197220a11644109953ed7947ccd908 (table test-workload [id=050b82e745d14eb5ac7792e930ab229a]) successfully deleted
I20251028 21:10:25.212188 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:25.212308 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:25.212334 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:27.212560 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:27.212671 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:27.212692 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:29.212837 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:29.212955 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:29.212980 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:31.213213 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:31.213338 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:31.213363 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:33.213616 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:33.213737 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:33.213757 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:35.213930 17874 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:10:35.214076 17874 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:10:35.214112 17874 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:35.877676 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:10:35.880337 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:35.880450 16180 tablet_replica.cc:333] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b: stopping tablet replica
I20251028 21:10:35.880510 16180 raft_consensus.cc:2243] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:35.880559 16180 raft_consensus.cc:2272] T cfad4b23af4f4f08ba66c40649131e63 P 5a5736535d4846cf88bc0c691730894b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:35.880722 16180 tablet_replica.cc:333] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b: stopping tablet replica
I20251028 21:10:35.880751 16180 raft_consensus.cc:2243] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:35.880805 16180 raft_consensus.cc:2272] T 7c197220a11644109953ed7947ccd908 P 5a5736535d4846cf88bc0c691730894b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:35.893132 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:10:35.894001 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:10:35.896767 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:35.896903 16180 tablet_replica.cc:333] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19: stopping tablet replica
I20251028 21:10:35.896957 16180 raft_consensus.cc:2243] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:35.897001 16180 raft_consensus.cc:2272] T cfad4b23af4f4f08ba66c40649131e63 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:35.897121 16180 tablet_replica.cc:333] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19: stopping tablet replica
I20251028 21:10:35.897153 16180 raft_consensus.cc:2243] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:35.897221 16180 raft_consensus.cc:2272] T 7c197220a11644109953ed7947ccd908 P 145ca98f4dbc4e30aa7330ab56807f19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:35.908440 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:10:35.909150 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:10:35.912290 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:35.912405 16180 tablet_replica.cc:333] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915: stopping tablet replica
I20251028 21:10:35.912448 16180 raft_consensus.cc:2243] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:35.912506 16180 raft_consensus.cc:2272] T cfad4b23af4f4f08ba66c40649131e63 P 8bed3b51c142463a93a29d0001d0a915 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:35.923799 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:10:35.924578 16180 tablet_server.cc:178] TabletServer@127.15.205.4:0 shutting down...
I20251028 21:10:35.927332 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:10:35.927424 16180 tablet_replica.cc:333] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0: stopping tablet replica
I20251028 21:10:35.927481 16180 raft_consensus.cc:2243] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:10:35.927531 16180 raft_consensus.cc:2272] T 7c197220a11644109953ed7947ccd908 P eb553c81bf674f0a94f810cb615e33c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:35.939663 16180 tablet_server.cc:195] TabletServer@127.15.205.4:0 shutdown complete.
I20251028 21:10:35.940359 16180 master.cc:561] Master@127.15.205.62:45519 shutting down...
I20251028 21:10:35.943084 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:35.943140 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:35.943158 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3fb90eb36603473ebe36c7dd73478537: stopping tablet replica
I20251028 21:10:35.954576 16180 master.cc:583] Master@127.15.205.62:45519 shutdown complete.
[ OK ] AutoRebalancerTest.MovesScheduledIfAddTserver (26773 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTservers
I20251028 21:10:35.959973 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:45979
I20251028 21:10:35.960089 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:35.961508 18268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:35.961622 18269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:35.961582 16180 server_base.cc:1047] running on GCE node
W20251028 21:10:35.961522 18271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:35.961836 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:35.961871 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:35.961884 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685835961884 us; error 0 us; skew 500 ppm
I20251028 21:10:35.962394 16180 webserver.cc:492] Webserver started at http://127.15.205.62:35189/ using document root <none> and password file <none>
I20251028 21:10:35.962453 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:35.962482 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:35.962513 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:35.962713 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "d79d68c47dba4f9ab7cd4102cd764d4c"
format_stamp: "Formatted at 2025-10-28 21:10:35 on dist-test-slave-zmkx"
I20251028 21:10:35.963514 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:10:35.963909 18276 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:35.964037 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:35.964090 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "d79d68c47dba4f9ab7cd4102cd764d4c"
format_stamp: "Formatted at 2025-10-28 21:10:35 on dist-test-slave-zmkx"
I20251028 21:10:35.964133 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:35.970587 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:35.970726 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:35.972823 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:45979
I20251028 21:10:35.974841 18328 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:45979 every 8 connection(s)
I20251028 21:10:35.974932 18329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:35.975961 18329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c: Bootstrap starting.
I20251028 21:10:35.976215 18329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:35.976631 18329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c: No bootstrap required, opened a new log
I20251028 21:10:35.976734 18329 raft_consensus.cc:359] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79d68c47dba4f9ab7cd4102cd764d4c" member_type: VOTER }
I20251028 21:10:35.976774 18329 raft_consensus.cc:385] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:35.976787 18329 raft_consensus.cc:740] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79d68c47dba4f9ab7cd4102cd764d4c, State: Initialized, Role: FOLLOWER
I20251028 21:10:35.976832 18329 consensus_queue.cc:260] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79d68c47dba4f9ab7cd4102cd764d4c" member_type: VOTER }
I20251028 21:10:35.976867 18329 raft_consensus.cc:399] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:10:35.976886 18329 raft_consensus.cc:493] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:10:35.976902 18329 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:35.977329 18329 raft_consensus.cc:515] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79d68c47dba4f9ab7cd4102cd764d4c" member_type: VOTER }
I20251028 21:10:35.977375 18329 leader_election.cc:304] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d79d68c47dba4f9ab7cd4102cd764d4c; no voters:
I20251028 21:10:35.977470 18329 leader_election.cc:290] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:10:35.977519 18332 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:35.977591 18329 sys_catalog.cc:565] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:10:35.977635 18332 raft_consensus.cc:697] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 1 LEADER]: Becoming Leader. State: Replica: d79d68c47dba4f9ab7cd4102cd764d4c, State: Running, Role: LEADER
I20251028 21:10:35.977675 18332 consensus_queue.cc:237] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79d68c47dba4f9ab7cd4102cd764d4c" member_type: VOTER }
I20251028 21:10:35.977833 18333 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d79d68c47dba4f9ab7cd4102cd764d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79d68c47dba4f9ab7cd4102cd764d4c" member_type: VOTER } }
I20251028 21:10:35.977882 18333 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:35.978006 18337 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:10:35.977988 18334 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d79d68c47dba4f9ab7cd4102cd764d4c. Latest consensus state: current_term: 1 leader_uuid: "d79d68c47dba4f9ab7cd4102cd764d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79d68c47dba4f9ab7cd4102cd764d4c" member_type: VOTER } }
I20251028 21:10:35.978128 18334 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:35.978173 18337 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:10:35.978775 18337 catalog_manager.cc:1357] Generated new cluster ID: 4f47ddc8a51b4660b6599b9e2b1d6831
I20251028 21:10:35.978861 18337 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:10:35.978873 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:10:35.993353 18337 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:10:35.993723 18337 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:10:36.002667 18337 catalog_manager.cc:6022] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c: Generated new TSK 0
I20251028 21:10:36.002728 18337 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:10:36.010481 16180 internal_mini_cluster.cc:371] 0 TS(s) registered with all masters after 3.3405e-05s
I20251028 21:10:48.086547 16180 master.cc:561] Master@127.15.205.62:45979 shutting down...
I20251028 21:10:48.088586 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:10:48.088654 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:10:48.088675 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P d79d68c47dba4f9ab7cd4102cd764d4c: stopping tablet replica
I20251028 21:10:48.099886 16180 master.cc:583] Master@127.15.205.62:45979 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTservers (12142 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTablets
I20251028 21:10:48.101979 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:44035
I20251028 21:10:48.102111 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:48.103279 18354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:48.103348 16180 server_base.cc:1047] running on GCE node
W20251028 21:10:48.103434 18351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:48.103487 18352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:48.103595 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:48.103632 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:48.103653 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685848103653 us; error 0 us; skew 500 ppm
I20251028 21:10:48.104281 16180 webserver.cc:492] Webserver started at http://127.15.205.62:45507/ using document root <none> and password file <none>
I20251028 21:10:48.104362 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:48.104404 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:48.104449 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:48.104682 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "70b87ab3bded4c3e97ce71f45cdcaf20"
format_stamp: "Formatted at 2025-10-28 21:10:48 on dist-test-slave-zmkx"
I20251028 21:10:48.105557 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:48.106045 18359 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.108184 16180 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20251028 21:10:48.108242 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "70b87ab3bded4c3e97ce71f45cdcaf20"
format_stamp: "Formatted at 2025-10-28 21:10:48 on dist-test-slave-zmkx"
I20251028 21:10:48.108285 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:48.128713 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:48.128899 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:48.132078 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:44035
I20251028 21:10:48.132740 18411 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:44035 every 8 connection(s)
I20251028 21:10:48.132943 18412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:10:48.134567 18412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20: Bootstrap starting.
I20251028 21:10:48.134919 18412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20: Neither blocks nor log segments found. Creating new log.
I20251028 21:10:48.135541 18412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20: No bootstrap required, opened a new log
I20251028 21:10:48.135707 18412 raft_consensus.cc:359] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b87ab3bded4c3e97ce71f45cdcaf20" member_type: VOTER }
I20251028 21:10:48.135797 18412 raft_consensus.cc:385] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:10:48.135852 18412 raft_consensus.cc:740] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70b87ab3bded4c3e97ce71f45cdcaf20, State: Initialized, Role: FOLLOWER
I20251028 21:10:48.135960 18412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b87ab3bded4c3e97ce71f45cdcaf20" member_type: VOTER }
I20251028 21:10:48.136039 18412 raft_consensus.cc:399] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:10:48.136104 18412 raft_consensus.cc:493] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:10:48.136168 18412 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:10:48.136940 18412 raft_consensus.cc:515] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b87ab3bded4c3e97ce71f45cdcaf20" member_type: VOTER }
I20251028 21:10:48.137065 18412 leader_election.cc:304] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70b87ab3bded4c3e97ce71f45cdcaf20; no voters:
I20251028 21:10:48.137223 18412 leader_election.cc:290] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:10:48.137285 18415 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:10:48.137363 18412 sys_catalog.cc:565] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:10:48.137418 18415 raft_consensus.cc:697] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 1 LEADER]: Becoming Leader. State: Replica: 70b87ab3bded4c3e97ce71f45cdcaf20, State: Running, Role: LEADER
I20251028 21:10:48.137462 18415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b87ab3bded4c3e97ce71f45cdcaf20" member_type: VOTER }
I20251028 21:10:48.137634 18416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70b87ab3bded4c3e97ce71f45cdcaf20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b87ab3bded4c3e97ce71f45cdcaf20" member_type: VOTER } }
I20251028 21:10:48.137691 18416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:48.137801 18417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70b87ab3bded4c3e97ce71f45cdcaf20. Latest consensus state: current_term: 1 leader_uuid: "70b87ab3bded4c3e97ce71f45cdcaf20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70b87ab3bded4c3e97ce71f45cdcaf20" member_type: VOTER } }
I20251028 21:10:48.137820 18420 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:10:48.137850 18417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [sys.catalog]: This master's current role is: LEADER
I20251028 21:10:48.137984 18420 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:10:48.138657 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:10:48.138664 18420 catalog_manager.cc:1357] Generated new cluster ID: 04edfdcbaa5347f5bfd6e6fb22b56ec1
I20251028 21:10:48.138715 18420 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:10:48.159586 18420 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:10:48.159936 18420 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:10:48.166219 18420 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20: Generated new TSK 0
I20251028 21:10:48.166288 18420 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:10:48.170444 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:48.171526 18434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:48.171672 16180 server_base.cc:1047] running on GCE node
W20251028 21:10:48.171564 18436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:48.171626 18433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:48.171840 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:48.171890 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:48.171903 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685848171904 us; error 0 us; skew 500 ppm
I20251028 21:10:48.172353 16180 webserver.cc:492] Webserver started at http://127.15.205.1:33323/ using document root <none> and password file <none>
I20251028 21:10:48.172410 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:48.172439 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:48.172472 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:48.172660 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "9a5b1e49bd38490ab40decc99b1f5ddd"
format_stamp: "Formatted at 2025-10-28 21:10:48 on dist-test-slave-zmkx"
I20251028 21:10:48.173393 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:10:48.173729 18441 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.173837 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.173880 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "9a5b1e49bd38490ab40decc99b1f5ddd"
format_stamp: "Formatted at 2025-10-28 21:10:48 on dist-test-slave-zmkx"
I20251028 21:10:48.173923 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:48.182444 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:48.182649 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:48.182786 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:48.182973 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:48.183000 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.183025 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:48.183054 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.185851 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:44365
I20251028 21:10:48.185896 18504 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:44365 every 8 connection(s)
I20251028 21:10:48.186200 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:48.187150 18509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:48.187196 18510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:48.187471 18512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:48.187664 16180 server_base.cc:1047] running on GCE node
I20251028 21:10:48.187767 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:48.187795 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:48.187820 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685848187820 us; error 0 us; skew 500 ppm
I20251028 21:10:48.188334 16180 webserver.cc:492] Webserver started at http://127.15.205.2:43529/ using document root <none> and password file <none>
I20251028 21:10:48.188411 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:48.188453 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:48.188539 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:48.188779 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "7538d844323245eabd0079e4fb382784"
format_stamp: "Formatted at 2025-10-28 21:10:48 on dist-test-slave-zmkx"
I20251028 21:10:48.189417 18505 heartbeater.cc:344] Connected to a master server at 127.15.205.62:44035
I20251028 21:10:48.189473 18505 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:48.189570 18505 heartbeater.cc:507] Master 127.15.205.62:44035 requested a full tablet report, sending...
I20251028 21:10:48.189770 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:10:48.189783 18376 ts_manager.cc:194] Registered new tserver with Master: 9a5b1e49bd38490ab40decc99b1f5ddd (127.15.205.1:44365)
I20251028 21:10:48.190228 18517 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.190289 18376 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46684
I20251028 21:10:48.190328 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.190364 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "7538d844323245eabd0079e4fb382784"
format_stamp: "Formatted at 2025-10-28 21:10:48 on dist-test-slave-zmkx"
I20251028 21:10:48.190402 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:48.208446 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:48.208580 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:48.208669 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:48.208789 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:48.208820 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.208847 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:48.208868 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.211453 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:44265
I20251028 21:10:48.211480 18580 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:44265 every 8 connection(s)
I20251028 21:10:48.211756 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:10:48.212802 18585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:10:48.212857 18584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:48.212934 16180 server_base.cc:1047] running on GCE node
W20251028 21:10:48.213022 18587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:10:48.213227 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:10:48.213260 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:10:48.213274 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685848213275 us; error 0 us; skew 500 ppm
I20251028 21:10:48.213773 16180 webserver.cc:492] Webserver started at http://127.15.205.3:38907/ using document root <none> and password file <none>
I20251028 21:10:48.213847 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:10:48.213886 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:10:48.213925 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:10:48.214190 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "c33809532df4427ba4b1cb11f51184eb"
format_stamp: "Formatted at 2025-10-28 21:10:48 on dist-test-slave-zmkx"
I20251028 21:10:48.214659 18581 heartbeater.cc:344] Connected to a master server at 127.15.205.62:44035
I20251028 21:10:48.214717 18581 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:48.214805 18581 heartbeater.cc:507] Master 127.15.205.62:44035 requested a full tablet report, sending...
I20251028 21:10:48.215049 18376 ts_manager.cc:194] Registered new tserver with Master: 7538d844323245eabd0079e4fb382784 (127.15.205.2:44265)
I20251028 21:10:48.215165 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:10:48.215561 18376 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46690
I20251028 21:10:48.215673 18592 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.215767 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.215809 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "c33809532df4427ba4b1cb11f51184eb"
format_stamp: "Formatted at 2025-10-28 21:10:48 on dist-test-slave-zmkx"
I20251028 21:10:48.215844 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:10:48.236418 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:10:48.236610 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:10:48.236745 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:10:48.236920 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:10:48.236948 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.236975 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:10:48.236997 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:10:48.240022 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:45065
I20251028 21:10:48.240078 18655 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:45065 every 8 connection(s)
I20251028 21:10:48.242372 18656 heartbeater.cc:344] Connected to a master server at 127.15.205.62:44035
I20251028 21:10:48.242414 18656 heartbeater.cc:461] Registering TS with master...
I20251028 21:10:48.242489 18656 heartbeater.cc:507] Master 127.15.205.62:44035 requested a full tablet report, sending...
I20251028 21:10:48.242681 18376 ts_manager.cc:194] Registered new tserver with Master: c33809532df4427ba4b1cb11f51184eb (127.15.205.3:45065)
I20251028 21:10:48.242982 18376 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46704
I20251028 21:10:48.243427 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003246001s
I20251028 21:10:49.190779 18505 heartbeater.cc:499] Master 127.15.205.62:44035 was elected leader, sending a full tablet report...
I20251028 21:10:49.216121 18581 heartbeater.cc:499] Master 127.15.205.62:44035 was elected leader, sending a full tablet report...
I20251028 21:10:49.243515 18656 heartbeater.cc:499] Master 127.15.205.62:44035 was elected leader, sending a full tablet report...
I20251028 21:10:50.138621 18430 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:52.138814 18430 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:54.139016 18430 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:56.139230 18430 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:10:58.139441 18430 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:00.139634 18430 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:00.323654 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:11:00.326552 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:00.337849 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:11:00.338668 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:11:00.341212 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:00.352329 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:11:00.352921 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:11:00.355515 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:00.366679 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:11:00.367277 16180 master.cc:561] Master@127.15.205.62:44035 shutting down...
I20251028 21:11:00.369441 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:11:00.369503 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:00.369558 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 70b87ab3bded4c3e97ce71f45cdcaf20: stopping tablet replica
I20251028 21:11:00.381273 16180 master.cc:583] Master@127.15.205.62:44035 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTablets (12282 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne
I20251028 21:11:00.384466 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:40857
I20251028 21:11:00.384599 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:00.385876 18660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:00.385970 16180 server_base.cc:1047] running on GCE node
W20251028 21:11:00.385987 18661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:00.385919 18663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:00.386190 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:00.386225 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:00.386247 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685860386247 us; error 0 us; skew 500 ppm
I20251028 21:11:00.386781 16180 webserver.cc:492] Webserver started at http://127.15.205.62:38657/ using document root <none> and password file <none>
I20251028 21:11:00.386861 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:00.386914 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:00.386962 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:00.387210 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "60a6f55dad2a4c2595d388b85c0ea232"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.388100 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:00.388577 18668 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.388696 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.388739 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "60a6f55dad2a4c2595d388b85c0ea232"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.388775 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:00.399151 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:00.399307 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:00.401456 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:40857
I20251028 21:11:00.402726 18720 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:40857 every 8 connection(s)
I20251028 21:11:00.402853 18721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:00.403751 18721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232: Bootstrap starting.
I20251028 21:11:00.404036 18721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:00.404425 18721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232: No bootstrap required, opened a new log
I20251028 21:11:00.404546 18721 raft_consensus.cc:359] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a6f55dad2a4c2595d388b85c0ea232" member_type: VOTER }
I20251028 21:11:00.404587 18721 raft_consensus.cc:385] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:00.404608 18721 raft_consensus.cc:740] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a6f55dad2a4c2595d388b85c0ea232, State: Initialized, Role: FOLLOWER
I20251028 21:11:00.404656 18721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a6f55dad2a4c2595d388b85c0ea232" member_type: VOTER }
I20251028 21:11:00.404692 18721 raft_consensus.cc:399] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:11:00.404712 18721 raft_consensus.cc:493] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:11:00.404738 18721 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:00.405148 18721 raft_consensus.cc:515] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a6f55dad2a4c2595d388b85c0ea232" member_type: VOTER }
I20251028 21:11:00.405200 18721 leader_election.cc:304] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60a6f55dad2a4c2595d388b85c0ea232; no voters:
I20251028 21:11:00.405292 18721 leader_election.cc:290] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:11:00.405334 18724 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:00.405417 18721 sys_catalog.cc:565] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:11:00.405454 18724 raft_consensus.cc:697] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 1 LEADER]: Becoming Leader. State: Replica: 60a6f55dad2a4c2595d388b85c0ea232, State: Running, Role: LEADER
I20251028 21:11:00.405506 18724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a6f55dad2a4c2595d388b85c0ea232" member_type: VOTER }
I20251028 21:11:00.405710 18725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60a6f55dad2a4c2595d388b85c0ea232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a6f55dad2a4c2595d388b85c0ea232" member_type: VOTER } }
I20251028 21:11:00.405881 18725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [sys.catalog]: This master's current role is: LEADER
I20251028 21:11:00.405727 18726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60a6f55dad2a4c2595d388b85c0ea232. Latest consensus state: current_term: 1 leader_uuid: "60a6f55dad2a4c2595d388b85c0ea232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60a6f55dad2a4c2595d388b85c0ea232" member_type: VOTER } }
I20251028 21:11:00.406024 18726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [sys.catalog]: This master's current role is: LEADER
I20251028 21:11:00.406087 18732 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:11:00.406275 18732 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:11:00.406723 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:11:00.406749 18732 catalog_manager.cc:1357] Generated new cluster ID: 23c3f4e158e44617865d379d75607ccb
I20251028 21:11:00.406776 18732 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:11:00.413462 18732 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:11:00.413816 18732 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:11:00.416390 18732 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232: Generated new TSK 0
I20251028 21:11:00.416448 18732 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:11:00.422318 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:00.423396 18743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:00.423404 18742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:00.423525 16180 server_base.cc:1047] running on GCE node
W20251028 21:11:00.423547 18745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:00.423657 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:00.423691 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:00.423707 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685860423708 us; error 0 us; skew 500 ppm
I20251028 21:11:00.424172 16180 webserver.cc:492] Webserver started at http://127.15.205.1:41109/ using document root <none> and password file <none>
I20251028 21:11:00.424249 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:00.424288 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:00.424328 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:00.424530 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "8d48f61cb448462f947e5d871f5a1829"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.425254 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:00.425599 18750 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.425697 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.425736 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "8d48f61cb448462f947e5d871f5a1829"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.425776 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:00.437431 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:00.437602 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:00.437721 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:00.437876 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:00.437912 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.437950 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:00.437976 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.440459 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:33417
I20251028 21:11:00.440487 18813 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:33417 every 8 connection(s)
I20251028 21:11:00.440755 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:00.441594 18818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:00.441782 18819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:00.441872 16180 server_base.cc:1047] running on GCE node
W20251028 21:11:00.441944 18821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:00.442065 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:00.442109 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:00.442122 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685860442122 us; error 0 us; skew 500 ppm
I20251028 21:11:00.442600 16180 webserver.cc:492] Webserver started at http://127.15.205.2:37451/ using document root <none> and password file <none>
I20251028 21:11:00.442669 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:00.442715 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:00.442760 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:00.443032 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "a7f6d03621b049c09f519c1534094989"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.443976 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:00.444449 18826 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.444576 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251028 21:11:00.444618 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "a7f6d03621b049c09f519c1534094989"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.444662 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:00.447676 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:00.447857 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:00.448009 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:00.448223 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:00.448275 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.448287 18814 heartbeater.cc:344] Connected to a master server at 127.15.205.62:40857
I20251028 21:11:00.448300 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:00.448354 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.448359 18814 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:00.448485 18814 heartbeater.cc:507] Master 127.15.205.62:40857 requested a full tablet report, sending...
I20251028 21:11:00.448683 18685 ts_manager.cc:194] Registered new tserver with Master: 8d48f61cb448462f947e5d871f5a1829 (127.15.205.1:33417)
I20251028 21:11:00.449388 18685 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52632
I20251028 21:11:00.451195 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:41235
I20251028 21:11:00.451225 18889 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:41235 every 8 connection(s)
I20251028 21:11:00.451496 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:00.452543 18893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:00.452562 18896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:00.452620 18894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:00.453220 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:00.453346 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:00.453385 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:00.453403 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685860453404 us; error 0 us; skew 500 ppm
I20251028 21:11:00.454231 16180 webserver.cc:492] Webserver started at http://127.15.205.3:43647/ using document root <none> and password file <none>
I20251028 21:11:00.454310 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:00.454356 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:00.454401 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:00.454419 18890 heartbeater.cc:344] Connected to a master server at 127.15.205.62:40857
I20251028 21:11:00.454488 18890 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:00.454581 18890 heartbeater.cc:507] Master 127.15.205.62:40857 requested a full tablet report, sending...
I20251028 21:11:00.454674 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "0b304b1c4b074f8c8033888751a97f97"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.454777 18685 ts_manager.cc:194] Registered new tserver with Master: a7f6d03621b049c09f519c1534094989 (127.15.205.2:41235)
I20251028 21:11:00.455083 18685 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52640
I20251028 21:11:00.455542 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:00.455998 18901 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.456120 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.456161 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "0b304b1c4b074f8c8033888751a97f97"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.456197 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:00.464766 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:00.464939 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:00.465060 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:00.465211 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:00.465237 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.465262 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:00.465289 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.468253 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:33325
I20251028 21:11:00.468303 18964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:33325 every 8 connection(s)
I20251028 21:11:00.468596 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:00.469532 18968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:00.469595 18971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:00.469755 18969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:00.469877 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:00.469980 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:00.470008 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:00.470023 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685860470024 us; error 0 us; skew 500 ppm
I20251028 21:11:00.470497 16180 webserver.cc:492] Webserver started at http://127.15.205.4:34325/ using document root <none> and password file <none>
I20251028 21:11:00.470578 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:00.470628 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:00.470671 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:00.470954 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-3-root/instance:
uuid: "5a527760b99b4b159823ce227d15d68d"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.471356 18965 heartbeater.cc:344] Connected to a master server at 127.15.205.62:40857
I20251028 21:11:00.471428 18965 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:00.471526 18965 heartbeater.cc:507] Master 127.15.205.62:40857 requested a full tablet report, sending...
I20251028 21:11:00.471697 18685 ts_manager.cc:194] Registered new tserver with Master: 0b304b1c4b074f8c8033888751a97f97 (127.15.205.3:33325)
I20251028 21:11:00.471971 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:00.472083 18685 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52646
I20251028 21:11:00.472365 18976 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.472458 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.472500 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-3-root
uuid: "5a527760b99b4b159823ce227d15d68d"
format_stamp: "Formatted at 2025-10-28 21:11:00 on dist-test-slave-zmkx"
I20251028 21:11:00.472537 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1761685792944951-16180-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:00.476609 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:00.476809 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:00.476924 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:00.477077 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:00.477102 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.477169 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:00.477195 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:00.480156 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.4:44927
I20251028 21:11:00.480196 19039 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.4:44927 every 8 connection(s)
I20251028 21:11:00.482473 19040 heartbeater.cc:344] Connected to a master server at 127.15.205.62:40857
I20251028 21:11:00.482517 19040 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:00.482609 19040 heartbeater.cc:507] Master 127.15.205.62:40857 requested a full tablet report, sending...
I20251028 21:11:00.482795 18685 ts_manager.cc:194] Registered new tserver with Master: 5a527760b99b4b159823ce227d15d68d (127.15.205.4:44927)
I20251028 21:11:00.483059 18685 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52648
I20251028 21:11:00.483565 16180 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.00328183s
I20251028 21:11:01.449990 18814 heartbeater.cc:499] Master 127.15.205.62:40857 was elected leader, sending a full tablet report...
I20251028 21:11:01.455585 18890 heartbeater.cc:499] Master 127.15.205.62:40857 was elected leader, sending a full tablet report...
I20251028 21:11:01.472543 18965 heartbeater.cc:499] Master 127.15.205.62:40857 was elected leader, sending a full tablet report...
I20251028 21:11:01.483568 19040 heartbeater.cc:499] Master 127.15.205.62:40857 was elected leader, sending a full tablet report...
I20251028 21:11:01.511148 16180 test_util.cc:276] Using random seed: 1945841643
I20251028 21:11:01.515471 18685 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52662:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 21:11:01.520176 18854 tablet_service.cc:1505] Processing CreateTablet for tablet f0981f83bb7748b6a9803603ac301490 (DEFAULT_TABLE table=test-workload [id=ac64645a30f24663817b6a5c01d18e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:11:01.520315 18854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0981f83bb7748b6a9803603ac301490. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:01.520740 18929 tablet_service.cc:1505] Processing CreateTablet for tablet f0981f83bb7748b6a9803603ac301490 (DEFAULT_TABLE table=test-workload [id=ac64645a30f24663817b6a5c01d18e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:11:01.520834 18929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0981f83bb7748b6a9803603ac301490. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:01.521659 19061 tablet_bootstrap.cc:492] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989: Bootstrap starting.
I20251028 21:11:01.522058 19062 tablet_bootstrap.cc:492] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97: Bootstrap starting.
I20251028 21:11:01.522074 19004 tablet_service.cc:1505] Processing CreateTablet for tablet f0981f83bb7748b6a9803603ac301490 (DEFAULT_TABLE table=test-workload [id=ac64645a30f24663817b6a5c01d18e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:11:01.522118 19061 tablet_bootstrap.cc:654] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:01.522207 19004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0981f83bb7748b6a9803603ac301490. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:01.522550 19062 tablet_bootstrap.cc:654] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:01.523070 19062 tablet_bootstrap.cc:492] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97: No bootstrap required, opened a new log
I20251028 21:11:01.523118 19062 ts_tablet_manager.cc:1403] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:01.523254 19065 tablet_bootstrap.cc:492] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d: Bootstrap starting.
I20251028 21:11:01.523299 19062 raft_consensus.cc:359] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.523430 19062 raft_consensus.cc:385] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:01.523484 19062 raft_consensus.cc:740] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b304b1c4b074f8c8033888751a97f97, State: Initialized, Role: FOLLOWER
I20251028 21:11:01.523519 19061 tablet_bootstrap.cc:492] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989: No bootstrap required, opened a new log
I20251028 21:11:01.523556 19061 ts_tablet_manager.cc:1403] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:11:01.523559 19062 consensus_queue.cc:260] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.523713 19061 raft_consensus.cc:359] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.523736 19062 ts_tablet_manager.cc:1434] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:01.523743 19065 tablet_bootstrap.cc:654] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:01.523764 19061 raft_consensus.cc:385] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:01.523842 19061 raft_consensus.cc:740] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f6d03621b049c09f519c1534094989, State: Initialized, Role: FOLLOWER
I20251028 21:11:01.523901 19061 consensus_queue.cc:260] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.524042 19061 ts_tablet_manager.cc:1434] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:01.524467 19065 tablet_bootstrap.cc:492] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d: No bootstrap required, opened a new log
I20251028 21:11:01.524516 19065 ts_tablet_manager.cc:1403] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:01.524677 19065 raft_consensus.cc:359] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.524736 19065 raft_consensus.cc:385] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:01.524765 19065 raft_consensus.cc:740] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a527760b99b4b159823ce227d15d68d, State: Initialized, Role: FOLLOWER
I20251028 21:11:01.524816 19065 consensus_queue.cc:260] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.524961 19065 ts_tablet_manager.cc:1434] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20251028 21:11:01.702251 18891 tablet.cc:2378] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251028 21:11:01.841367 19066 raft_consensus.cc:493] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:01.841450 19066 raft_consensus.cc:515] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.841624 19066 leader_election.cc:290] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a527760b99b4b159823ce227d15d68d (127.15.205.4:44927), a7f6d03621b049c09f519c1534094989 (127.15.205.2:41235)
I20251028 21:11:01.844139 18864 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f0981f83bb7748b6a9803603ac301490" candidate_uuid: "0b304b1c4b074f8c8033888751a97f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7f6d03621b049c09f519c1534094989" is_pre_election: true
I20251028 21:11:01.844151 19014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f0981f83bb7748b6a9803603ac301490" candidate_uuid: "0b304b1c4b074f8c8033888751a97f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a527760b99b4b159823ce227d15d68d" is_pre_election: true
I20251028 21:11:01.844234 18864 raft_consensus.cc:2468] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b304b1c4b074f8c8033888751a97f97 in term 0.
I20251028 21:11:01.844246 19014 raft_consensus.cc:2468] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b304b1c4b074f8c8033888751a97f97 in term 0.
I20251028 21:11:01.844357 18902 leader_election.cc:304] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b304b1c4b074f8c8033888751a97f97, a7f6d03621b049c09f519c1534094989; no voters:
I20251028 21:11:01.844439 19066 raft_consensus.cc:2804] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:01.844488 19066 raft_consensus.cc:493] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:01.844522 19066 raft_consensus.cc:3060] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:01.845078 19066 raft_consensus.cc:515] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.845178 19066 leader_election.cc:290] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [CANDIDATE]: Term 1 election: Requested vote from peers 5a527760b99b4b159823ce227d15d68d (127.15.205.4:44927), a7f6d03621b049c09f519c1534094989 (127.15.205.2:41235)
I20251028 21:11:01.845314 19014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f0981f83bb7748b6a9803603ac301490" candidate_uuid: "0b304b1c4b074f8c8033888751a97f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a527760b99b4b159823ce227d15d68d"
I20251028 21:11:01.845319 18864 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f0981f83bb7748b6a9803603ac301490" candidate_uuid: "0b304b1c4b074f8c8033888751a97f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7f6d03621b049c09f519c1534094989"
I20251028 21:11:01.845391 19014 raft_consensus.cc:3060] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:01.845398 18864 raft_consensus.cc:3060] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:01.846073 18864 raft_consensus.cc:2468] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b304b1c4b074f8c8033888751a97f97 in term 1.
I20251028 21:11:01.846073 19014 raft_consensus.cc:2468] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b304b1c4b074f8c8033888751a97f97 in term 1.
I20251028 21:11:01.846246 18905 leader_election.cc:304] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b304b1c4b074f8c8033888751a97f97, 5a527760b99b4b159823ce227d15d68d; no voters:
I20251028 21:11:01.846324 19066 raft_consensus.cc:2804] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:01.846460 19066 raft_consensus.cc:697] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 1 LEADER]: Becoming Leader. State: Replica: 0b304b1c4b074f8c8033888751a97f97, State: Running, Role: LEADER
I20251028 21:11:01.846505 19066 consensus_queue.cc:237] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } }
I20251028 21:11:01.846987 18684 catalog_manager.cc:5649] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b304b1c4b074f8c8033888751a97f97 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "0b304b1c4b074f8c8033888751a97f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b304b1c4b074f8c8033888751a97f97" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 33325 } health_report { overall_health: HEALTHY } } }
I20251028 21:11:02.395699 19074 consensus_queue.cc:1048] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7f6d03621b049c09f519c1534094989" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 41235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:02.397066 19074 consensus_queue.cc:1048] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a527760b99b4b159823ce227d15d68d" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 44927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:02.406733 18739 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:02.406819 18739 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:02.406850 18739 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:04.406970 18739 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:04.407097 18739 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:04.407137 18739 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:06.407295 18739 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:06.407404 18739 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:06.407440 18739 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:08.407570 18739 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:08.407686 18739 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:08.407717 18739 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:10.407851 18739 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:10.407950 18739 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:10.407976 18739 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:12.408087 18739 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:12.408188 18739 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:12.408226 18739 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:12.918744 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:11:12.921171 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:12.932459 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:11:12.933144 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:11:12.935750 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:12.935842 16180 tablet_replica.cc:333] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989: stopping tablet replica
I20251028 21:11:12.935895 16180 raft_consensus.cc:2243] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:12.935940 16180 raft_consensus.cc:2272] T f0981f83bb7748b6a9803603ac301490 P a7f6d03621b049c09f519c1534094989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:12.947383 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:11:12.948187 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:11:12.950513 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:12.950618 16180 tablet_replica.cc:333] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97: stopping tablet replica
I20251028 21:11:12.950672 16180 raft_consensus.cc:2243] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:11:12.950731 16180 raft_consensus.cc:2272] T f0981f83bb7748b6a9803603ac301490 P 0b304b1c4b074f8c8033888751a97f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:12.962085 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:11:12.962672 16180 tablet_server.cc:178] TabletServer@127.15.205.4:0 shutting down...
I20251028 21:11:12.964874 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:12.964967 16180 tablet_replica.cc:333] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d: stopping tablet replica
I20251028 21:11:12.965016 16180 raft_consensus.cc:2243] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:12.965054 16180 raft_consensus.cc:2272] T f0981f83bb7748b6a9803603ac301490 P 5a527760b99b4b159823ce227d15d68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:12.976227 16180 tablet_server.cc:195] TabletServer@127.15.205.4:0 shutdown complete.
I20251028 21:11:12.976944 16180 master.cc:561] Master@127.15.205.62:40857 shutting down...
I20251028 21:11:12.979245 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:11:12.979341 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:12.979364 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 60a6f55dad2a4c2595d388b85c0ea232: stopping tablet replica
I20251028 21:11:12.990687 16180 master.cc:583] Master@127.15.205.62:40857 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne (12610 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy
I20251028 21:11:12.994696 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:40375
I20251028 21:11:12.994812 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:12.995929 19084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:12.995939 19087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:12.996050 19085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:12.996136 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:12.996227 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:12.996258 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:12.996284 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685872996284 us; error 0 us; skew 500 ppm
I20251028 21:11:12.996809 16180 webserver.cc:492] Webserver started at http://127.15.205.62:43163/ using document root <none> and password file <none>
I20251028 21:11:12.996894 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:12.996937 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:12.996979 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:12.997208 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "727553ce38e64d3f8b0f033590d27c35"
format_stamp: "Formatted at 2025-10-28 21:11:12 on dist-test-slave-zmkx"
I20251028 21:11:12.997963 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:12.998318 19092 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:12.998477 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:12.998518 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "727553ce38e64d3f8b0f033590d27c35"
format_stamp: "Formatted at 2025-10-28 21:11:12 on dist-test-slave-zmkx"
I20251028 21:11:12.998569 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:13.003773 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:13.003932 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:13.006145 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:40375
I20251028 21:11:13.008076 19144 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:40375 every 8 connection(s)
I20251028 21:11:13.008144 19145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:13.009047 19145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35: Bootstrap starting.
I20251028 21:11:13.009300 19145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:13.009725 19145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35: No bootstrap required, opened a new log
I20251028 21:11:13.009827 19145 raft_consensus.cc:359] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727553ce38e64d3f8b0f033590d27c35" member_type: VOTER }
I20251028 21:11:13.009873 19145 raft_consensus.cc:385] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:13.009896 19145 raft_consensus.cc:740] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 727553ce38e64d3f8b0f033590d27c35, State: Initialized, Role: FOLLOWER
I20251028 21:11:13.009946 19145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727553ce38e64d3f8b0f033590d27c35" member_type: VOTER }
I20251028 21:11:13.009980 19145 raft_consensus.cc:399] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:11:13.010000 19145 raft_consensus.cc:493] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:11:13.010026 19145 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:13.010437 19145 raft_consensus.cc:515] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727553ce38e64d3f8b0f033590d27c35" member_type: VOTER }
I20251028 21:11:13.010486 19145 leader_election.cc:304] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 727553ce38e64d3f8b0f033590d27c35; no voters:
I20251028 21:11:13.010573 19145 leader_election.cc:290] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:11:13.010615 19148 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:13.010691 19145 sys_catalog.cc:565] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:11:13.010766 19148 raft_consensus.cc:697] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 1 LEADER]: Becoming Leader. State: Replica: 727553ce38e64d3f8b0f033590d27c35, State: Running, Role: LEADER
I20251028 21:11:13.010816 19148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727553ce38e64d3f8b0f033590d27c35" member_type: VOTER }
I20251028 21:11:13.010968 19148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 727553ce38e64d3f8b0f033590d27c35. Latest consensus state: current_term: 1 leader_uuid: "727553ce38e64d3f8b0f033590d27c35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727553ce38e64d3f8b0f033590d27c35" member_type: VOTER } }
I20251028 21:11:13.011044 19148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [sys.catalog]: This master's current role is: LEADER
I20251028 21:11:13.011003 19149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "727553ce38e64d3f8b0f033590d27c35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "727553ce38e64d3f8b0f033590d27c35" member_type: VOTER } }
I20251028 21:11:13.011170 19149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [sys.catalog]: This master's current role is: LEADER
I20251028 21:11:13.011497 19157 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:11:13.011619 19157 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:11:13.011895 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:11:13.012146 19157 catalog_manager.cc:1357] Generated new cluster ID: a6221272c87a410db3d5d8d996416108
I20251028 21:11:13.012187 19157 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:11:13.022447 19157 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:11:13.022783 19157 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:11:13.032799 19157 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35: Generated new TSK 0
I20251028 21:11:13.032855 19157 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:11:13.043545 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:13.044642 19166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:13.044701 19169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:13.044792 19167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:13.044863 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:13.044963 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:13.044996 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:13.045012 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685873045013 us; error 0 us; skew 500 ppm
I20251028 21:11:13.045465 16180 webserver.cc:492] Webserver started at http://127.15.205.1:35287/ using document root <none> and password file <none>
I20251028 21:11:13.045527 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:13.045557 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:13.045588 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:13.045778 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "58002339bee6420aa7cb4d9a1eef4d65"
format_stamp: "Formatted at 2025-10-28 21:11:13 on dist-test-slave-zmkx"
I20251028 21:11:13.046564 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:13.046921 19174 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.047045 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.047083 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "58002339bee6420aa7cb4d9a1eef4d65"
format_stamp: "Formatted at 2025-10-28 21:11:13 on dist-test-slave-zmkx"
I20251028 21:11:13.047132 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:13.065924 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:13.066092 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:13.066205 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:13.066358 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:13.066382 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.066411 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:13.066434 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.068982 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:35199
I20251028 21:11:13.069008 19237 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:35199 every 8 connection(s)
I20251028 21:11:13.069294 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:13.070317 19245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:13.070343 19242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:13.070541 19243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:13.070555 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:13.070699 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:13.070727 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:13.070752 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685873070753 us; error 0 us; skew 500 ppm
I20251028 21:11:13.071244 16180 webserver.cc:492] Webserver started at http://127.15.205.2:46697/ using document root <none> and password file <none>
I20251028 21:11:13.071331 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:13.071368 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:13.071408 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:13.071655 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "eaa469293e564271ba740e9da876e8a0"
format_stamp: "Formatted at 2025-10-28 21:11:13 on dist-test-slave-zmkx"
I20251028 21:11:13.072463 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:13.072808 19250 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.072902 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.072947 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "eaa469293e564271ba740e9da876e8a0"
format_stamp: "Formatted at 2025-10-28 21:11:13 on dist-test-slave-zmkx"
I20251028 21:11:13.072988 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:13.075814 19238 heartbeater.cc:344] Connected to a master server at 127.15.205.62:40375
I20251028 21:11:13.075868 19238 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:13.075953 19238 heartbeater.cc:507] Master 127.15.205.62:40375 requested a full tablet report, sending...
I20251028 21:11:13.076134 19109 ts_manager.cc:194] Registered new tserver with Master: 58002339bee6420aa7cb4d9a1eef4d65 (127.15.205.1:35199)
I20251028 21:11:13.076618 19109 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50702
I20251028 21:11:13.083324 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:13.083470 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:13.083580 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:13.083732 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:13.083757 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.083784 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:13.083813 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.086365 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:42267
I20251028 21:11:13.086400 19313 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:42267 every 8 connection(s)
I20251028 21:11:13.086673 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:13.087656 19317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:13.087750 19318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:13.088024 16180 server_base.cc:1047] running on GCE node
W20251028 21:11:13.087672 19320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:13.088294 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:13.088327 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:13.088356 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685873088357 us; error 0 us; skew 500 ppm
I20251028 21:11:13.088891 16180 webserver.cc:492] Webserver started at http://127.15.205.3:36545/ using document root <none> and password file <none>
I20251028 21:11:13.088975 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:13.089020 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:13.089067 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:13.089330 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "d3180b16e73946bb8632450398fdcc87"
format_stamp: "Formatted at 2025-10-28 21:11:13 on dist-test-slave-zmkx"
I20251028 21:11:13.089587 19314 heartbeater.cc:344] Connected to a master server at 127.15.205.62:40375
I20251028 21:11:13.089646 19314 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:13.089747 19314 heartbeater.cc:507] Master 127.15.205.62:40375 requested a full tablet report, sending...
I20251028 21:11:13.089938 19109 ts_manager.cc:194] Registered new tserver with Master: eaa469293e564271ba740e9da876e8a0 (127.15.205.2:42267)
I20251028 21:11:13.090277 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:13.090353 19109 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50708
I20251028 21:11:13.090729 19325 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.090852 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.090909 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "d3180b16e73946bb8632450398fdcc87"
format_stamp: "Formatted at 2025-10-28 21:11:13 on dist-test-slave-zmkx"
I20251028 21:11:13.090950 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:13.095304 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:13.095472 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:13.095573 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:13.095705 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:13.095726 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.095759 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:13.095772 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:13.098600 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:38577
I20251028 21:11:13.098654 19388 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:38577 every 8 connection(s)
I20251028 21:11:13.101200 19389 heartbeater.cc:344] Connected to a master server at 127.15.205.62:40375
I20251028 21:11:13.101253 19389 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:13.101332 19389 heartbeater.cc:507] Master 127.15.205.62:40375 requested a full tablet report, sending...
I20251028 21:11:13.101511 19109 ts_manager.cc:194] Registered new tserver with Master: d3180b16e73946bb8632450398fdcc87 (127.15.205.3:38577)
I20251028 21:11:13.101794 19109 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50724
I20251028 21:11:13.102075 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003303629s
I20251028 21:11:14.077064 19238 heartbeater.cc:499] Master 127.15.205.62:40375 was elected leader, sending a full tablet report...
I20251028 21:11:14.090931 19314 heartbeater.cc:499] Master 127.15.205.62:40375 was elected leader, sending a full tablet report...
I20251028 21:11:14.102276 19389 heartbeater.cc:499] Master 127.15.205.62:40375 was elected leader, sending a full tablet report...
I20251028 21:11:14.129804 16180 test_util.cc:276] Using random seed: 1958460299
I20251028 21:11:14.133822 19109 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50736:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:11:14.133927 19109 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:11:14.138159 19278 tablet_service.cc:1505] Processing CreateTablet for tablet f9dff920641a45d6bcf00d88ff9c5949 (DEFAULT_TABLE table=test-workload [id=af05075a39d0463596737f63223b75ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:11:14.138288 19278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9dff920641a45d6bcf00d88ff9c5949. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:14.138458 19353 tablet_service.cc:1505] Processing CreateTablet for tablet f9dff920641a45d6bcf00d88ff9c5949 (DEFAULT_TABLE table=test-workload [id=af05075a39d0463596737f63223b75ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:11:14.138535 19353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9dff920641a45d6bcf00d88ff9c5949. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:14.139766 19410 tablet_bootstrap.cc:492] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0: Bootstrap starting.
I20251028 21:11:14.139830 19202 tablet_service.cc:1505] Processing CreateTablet for tablet f9dff920641a45d6bcf00d88ff9c5949 (DEFAULT_TABLE table=test-workload [id=af05075a39d0463596737f63223b75ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:11:14.139935 19202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9dff920641a45d6bcf00d88ff9c5949. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:14.140200 19410 tablet_bootstrap.cc:654] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:14.140487 19411 tablet_bootstrap.cc:492] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87: Bootstrap starting.
I20251028 21:11:14.140767 19410 tablet_bootstrap.cc:492] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0: No bootstrap required, opened a new log
I20251028 21:11:14.140812 19410 ts_tablet_manager.cc:1403] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:14.140901 19413 tablet_bootstrap.cc:492] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65: Bootstrap starting.
I20251028 21:11:14.140904 19411 tablet_bootstrap.cc:654] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:14.140959 19410 raft_consensus.cc:359] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.141038 19410 raft_consensus.cc:385] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:14.141077 19410 raft_consensus.cc:740] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaa469293e564271ba740e9da876e8a0, State: Initialized, Role: FOLLOWER
I20251028 21:11:14.141139 19410 consensus_queue.cc:260] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.141310 19413 tablet_bootstrap.cc:654] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:14.141314 19410 ts_tablet_manager.cc:1434] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:11:14.141536 19411 tablet_bootstrap.cc:492] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87: No bootstrap required, opened a new log
I20251028 21:11:14.141573 19411 ts_tablet_manager.cc:1403] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:14.141701 19411 raft_consensus.cc:359] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.141744 19411 raft_consensus.cc:385] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:14.141767 19411 raft_consensus.cc:740] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3180b16e73946bb8632450398fdcc87, State: Initialized, Role: FOLLOWER
I20251028 21:11:14.141819 19411 consensus_queue.cc:260] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.141919 19413 tablet_bootstrap.cc:492] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65: No bootstrap required, opened a new log
I20251028 21:11:14.141964 19411 ts_tablet_manager.cc:1434] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:14.141992 19413 ts_tablet_manager.cc:1403] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:14.142131 19413 raft_consensus.cc:359] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.142302 19413 raft_consensus.cc:385] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:14.142339 19413 raft_consensus.cc:740] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58002339bee6420aa7cb4d9a1eef4d65, State: Initialized, Role: FOLLOWER
I20251028 21:11:14.142395 19413 consensus_queue.cc:260] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.142562 19413 ts_tablet_manager.cc:1434] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:11:14.387302 19417 raft_consensus.cc:493] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:14.387394 19417 raft_consensus.cc:515] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.387601 19417 leader_election.cc:290] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaa469293e564271ba740e9da876e8a0 (127.15.205.2:42267), 58002339bee6420aa7cb4d9a1eef4d65 (127.15.205.1:35199)
I20251028 21:11:14.390148 19288 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9dff920641a45d6bcf00d88ff9c5949" candidate_uuid: "d3180b16e73946bb8632450398fdcc87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaa469293e564271ba740e9da876e8a0" is_pre_election: true
I20251028 21:11:14.390242 19288 raft_consensus.cc:2468] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3180b16e73946bb8632450398fdcc87 in term 0.
I20251028 21:11:14.390331 19212 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9dff920641a45d6bcf00d88ff9c5949" candidate_uuid: "d3180b16e73946bb8632450398fdcc87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58002339bee6420aa7cb4d9a1eef4d65" is_pre_election: true
I20251028 21:11:14.390399 19328 leader_election.cc:304] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3180b16e73946bb8632450398fdcc87, eaa469293e564271ba740e9da876e8a0; no voters:
I20251028 21:11:14.390417 19212 raft_consensus.cc:2468] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3180b16e73946bb8632450398fdcc87 in term 0.
I20251028 21:11:14.390507 19417 raft_consensus.cc:2804] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:14.390547 19417 raft_consensus.cc:493] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:14.390571 19417 raft_consensus.cc:3060] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:14.391160 19417 raft_consensus.cc:515] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.391287 19417 leader_election.cc:290] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [CANDIDATE]: Term 1 election: Requested vote from peers eaa469293e564271ba740e9da876e8a0 (127.15.205.2:42267), 58002339bee6420aa7cb4d9a1eef4d65 (127.15.205.1:35199)
I20251028 21:11:14.391381 19212 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9dff920641a45d6bcf00d88ff9c5949" candidate_uuid: "d3180b16e73946bb8632450398fdcc87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58002339bee6420aa7cb4d9a1eef4d65"
I20251028 21:11:14.391400 19288 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9dff920641a45d6bcf00d88ff9c5949" candidate_uuid: "d3180b16e73946bb8632450398fdcc87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaa469293e564271ba740e9da876e8a0"
I20251028 21:11:14.391450 19212 raft_consensus.cc:3060] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:14.391462 19288 raft_consensus.cc:3060] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:14.392087 19212 raft_consensus.cc:2468] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3180b16e73946bb8632450398fdcc87 in term 1.
I20251028 21:11:14.392091 19288 raft_consensus.cc:2468] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3180b16e73946bb8632450398fdcc87 in term 1.
I20251028 21:11:14.392251 19327 leader_election.cc:304] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58002339bee6420aa7cb4d9a1eef4d65, d3180b16e73946bb8632450398fdcc87; no voters:
I20251028 21:11:14.392342 19417 raft_consensus.cc:2804] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:14.392470 19417 raft_consensus.cc:697] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 1 LEADER]: Becoming Leader. State: Replica: d3180b16e73946bb8632450398fdcc87, State: Running, Role: LEADER
I20251028 21:11:14.392521 19417 consensus_queue.cc:237] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } }
I20251028 21:11:14.393019 19109 catalog_manager.cc:5649] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3180b16e73946bb8632450398fdcc87 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "d3180b16e73946bb8632450398fdcc87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3180b16e73946bb8632450398fdcc87" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 38577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:14.899259 19423 consensus_queue.cc:1048] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaa469293e564271ba740e9da876e8a0" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 42267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:14.900652 19426 consensus_queue.cc:1048] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58002339bee6420aa7cb4d9a1eef4d65" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:15.011691 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:15.012048 19163 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:15.012157 19163 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:15.012197 19163 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:16.012189 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:17.012341 19163 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:17.012478 19163 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:17.012514 19163 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:17.012578 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:18.012907 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:19.012639 19163 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:19.012756 19163 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:19.012797 19163 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:19.013266 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:20.013698 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:21.012952 19163 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:21.013062 19163 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:21.013082 19163 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:21.014058 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:22.014514 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:23.013228 19163 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:23.013340 19163 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:23.013375 19163 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:23.014865 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:24.015187 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:25.013512 19163 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:25.013635 19163 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:25.013657 19163 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:25.015544 19156 placement_policy_util.cc:407] tablet f9dff920641a45d6bcf00d88ff9c5949: detected majority of replicas (2 of 3) at location L0
I20251028 21:11:25.467846 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:11:25.470219 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:25.470345 16180 tablet_replica.cc:333] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65: stopping tablet replica
I20251028 21:11:25.470414 16180 raft_consensus.cc:2243] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:25.470468 16180 raft_consensus.cc:2272] T f9dff920641a45d6bcf00d88ff9c5949 P 58002339bee6420aa7cb4d9a1eef4d65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:25.482168 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:11:25.482868 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:11:25.485059 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:25.485145 16180 tablet_replica.cc:333] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0: stopping tablet replica
I20251028 21:11:25.485189 16180 raft_consensus.cc:2243] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:25.485236 16180 raft_consensus.cc:2272] T f9dff920641a45d6bcf00d88ff9c5949 P eaa469293e564271ba740e9da876e8a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:25.496366 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:11:25.496951 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:11:25.498732 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:11:25.498813 16180 tablet_replica.cc:333] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87: stopping tablet replica
I20251028 21:11:25.498860 16180 raft_consensus.cc:2243] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:11:25.498919 16180 raft_consensus.cc:2272] T f9dff920641a45d6bcf00d88ff9c5949 P d3180b16e73946bb8632450398fdcc87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:25.510075 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:11:25.510743 16180 master.cc:561] Master@127.15.205.62:40375 shutting down...
I20251028 21:11:25.512694 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:11:25.512743 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:25.512759 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 727553ce38e64d3f8b0f033590d27c35: stopping tablet replica
I20251028 21:11:25.523825 16180 master.cc:583] Master@127.15.205.62:40375 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy (12532 ms)
[ RUN ] AutoRebalancerTest.TestMaxMovesPerServer
I20251028 21:11:25.527510 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:41741
I20251028 21:11:25.527638 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:25.528856 19434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:25.528913 19436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:25.529004 19433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:25.529042 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:25.529119 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:25.529148 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:25.529163 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685885529163 us; error 0 us; skew 500 ppm
I20251028 21:11:25.529666 16180 webserver.cc:492] Webserver started at http://127.15.205.62:41183/ using document root <none> and password file <none>
I20251028 21:11:25.529738 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:25.529771 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:25.529803 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:25.530019 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "47d74768e36048b29dc382adb5d0bf36"
format_stamp: "Formatted at 2025-10-28 21:11:25 on dist-test-slave-zmkx"
I20251028 21:11:25.530823 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:25.531204 19441 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.531396 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.531435 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "47d74768e36048b29dc382adb5d0bf36"
format_stamp: "Formatted at 2025-10-28 21:11:25 on dist-test-slave-zmkx"
I20251028 21:11:25.531467 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:25.561272 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:25.561393 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:25.563416 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:41741
I20251028 21:11:25.565062 19493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:41741 every 8 connection(s)
I20251028 21:11:25.565167 19494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:25.565958 19494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36: Bootstrap starting.
I20251028 21:11:25.566159 19494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:25.566535 19494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36: No bootstrap required, opened a new log
I20251028 21:11:25.566619 19494 raft_consensus.cc:359] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d74768e36048b29dc382adb5d0bf36" member_type: VOTER }
I20251028 21:11:25.566655 19494 raft_consensus.cc:385] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:25.566668 19494 raft_consensus.cc:740] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47d74768e36048b29dc382adb5d0bf36, State: Initialized, Role: FOLLOWER
I20251028 21:11:25.566699 19494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d74768e36048b29dc382adb5d0bf36" member_type: VOTER }
I20251028 21:11:25.566740 19494 raft_consensus.cc:399] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:11:25.566764 19494 raft_consensus.cc:493] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:11:25.566782 19494 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:25.567184 19494 raft_consensus.cc:515] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d74768e36048b29dc382adb5d0bf36" member_type: VOTER }
I20251028 21:11:25.567229 19494 leader_election.cc:304] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47d74768e36048b29dc382adb5d0bf36; no voters:
I20251028 21:11:25.567339 19494 leader_election.cc:290] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:11:25.567402 19497 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:25.567453 19494 sys_catalog.cc:565] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:11:25.567507 19497 raft_consensus.cc:697] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 1 LEADER]: Becoming Leader. State: Replica: 47d74768e36048b29dc382adb5d0bf36, State: Running, Role: LEADER
I20251028 21:11:25.567572 19497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d74768e36048b29dc382adb5d0bf36" member_type: VOTER }
I20251028 21:11:25.567757 19498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47d74768e36048b29dc382adb5d0bf36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d74768e36048b29dc382adb5d0bf36" member_type: VOTER } }
I20251028 21:11:25.567788 19499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47d74768e36048b29dc382adb5d0bf36. Latest consensus state: current_term: 1 leader_uuid: "47d74768e36048b29dc382adb5d0bf36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d74768e36048b29dc382adb5d0bf36" member_type: VOTER } }
I20251028 21:11:25.567885 19498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [sys.catalog]: This master's current role is: LEADER
I20251028 21:11:25.567919 19499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [sys.catalog]: This master's current role is: LEADER
I20251028 21:11:25.568123 19504 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:11:25.568255 19504 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:11:25.568692 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:11:25.568754 19504 catalog_manager.cc:1357] Generated new cluster ID: b58c1a5f3aa74f50bb84391742075fc3
I20251028 21:11:25.568787 19504 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:11:25.589332 19504 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:11:25.589695 19504 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:11:25.595950 19504 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36: Generated new TSK 0
I20251028 21:11:25.596004 19504 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:11:25.600349 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:25.601459 19516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:25.601538 19518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:25.601567 19515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:25.601730 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:25.601796 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:25.601828 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:25.601851 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685885601851 us; error 0 us; skew 500 ppm
I20251028 21:11:25.602324 16180 webserver.cc:492] Webserver started at http://127.15.205.1:40167/ using document root <none> and password file <none>
I20251028 21:11:25.602387 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:25.602430 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:25.602473 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:25.602702 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "95350159593b4057ae155343f06246d0"
format_stamp: "Formatted at 2025-10-28 21:11:25 on dist-test-slave-zmkx"
I20251028 21:11:25.603456 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:25.603824 19523 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.603932 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.603966 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "95350159593b4057ae155343f06246d0"
format_stamp: "Formatted at 2025-10-28 21:11:25 on dist-test-slave-zmkx"
I20251028 21:11:25.604000 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:25.612210 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:25.612361 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:25.612486 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:25.612648 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:25.612674 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.612699 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:25.612722 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.615108 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:38429
I20251028 21:11:25.615134 19586 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:38429 every 8 connection(s)
I20251028 21:11:25.615427 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:25.616374 19591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:25.616454 19592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:25.616472 19594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:25.616762 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:25.616885 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:25.616916 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:25.616935 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685885616935 us; error 0 us; skew 500 ppm
I20251028 21:11:25.617475 16180 webserver.cc:492] Webserver started at http://127.15.205.2:41671/ using document root <none> and password file <none>
I20251028 21:11:25.617547 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:25.617589 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:25.617628 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:25.617892 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
format_stamp: "Formatted at 2025-10-28 21:11:25 on dist-test-slave-zmkx"
I20251028 21:11:25.618865 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:25.618969 19587 heartbeater.cc:344] Connected to a master server at 127.15.205.62:41741
I20251028 21:11:25.619021 19587 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:25.619103 19587 heartbeater.cc:507] Master 127.15.205.62:41741 requested a full tablet report, sending...
I20251028 21:11:25.619315 19458 ts_manager.cc:194] Registered new tserver with Master: 95350159593b4057ae155343f06246d0 (127.15.205.1:38429)
I20251028 21:11:25.619346 19599 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.619422 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.619454 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
format_stamp: "Formatted at 2025-10-28 21:11:25 on dist-test-slave-zmkx"
I20251028 21:11:25.619477 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:25.619868 19458 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54996
I20251028 21:11:25.627705 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:25.627861 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:25.627971 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:25.628140 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:25.628166 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.628185 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:25.628214 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.630839 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:36457
I20251028 21:11:25.630867 19662 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:36457 every 8 connection(s)
I20251028 21:11:25.631127 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:25.632090 19666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:25.632157 19667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:25.632328 16180 server_base.cc:1047] running on GCE node
W20251028 21:11:25.632387 19669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:25.632527 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:25.632560 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:25.632576 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685885632576 us; error 0 us; skew 500 ppm
I20251028 21:11:25.633052 16180 webserver.cc:492] Webserver started at http://127.15.205.3:35943/ using document root <none> and password file <none>
I20251028 21:11:25.633122 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:25.633160 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:25.633199 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:25.633435 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
format_stamp: "Formatted at 2025-10-28 21:11:25 on dist-test-slave-zmkx"
I20251028 21:11:25.633648 19663 heartbeater.cc:344] Connected to a master server at 127.15.205.62:41741
I20251028 21:11:25.633697 19663 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:25.633775 19663 heartbeater.cc:507] Master 127.15.205.62:41741 requested a full tablet report, sending...
I20251028 21:11:25.633972 19458 ts_manager.cc:194] Registered new tserver with Master: 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:25.634244 19458 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55006
I20251028 21:11:25.634302 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:25.634688 19674 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.634791 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.634827 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
format_stamp: "Formatted at 2025-10-28 21:11:25 on dist-test-slave-zmkx"
I20251028 21:11:25.634864 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:25.653832 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:25.654019 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:25.654138 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:25.654282 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:25.654305 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.654331 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:25.654354 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:25.657073 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:42465
I20251028 21:11:25.657120 19737 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:42465 every 8 connection(s)
I20251028 21:11:25.659446 19738 heartbeater.cc:344] Connected to a master server at 127.15.205.62:41741
I20251028 21:11:25.659490 19738 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:25.659590 19738 heartbeater.cc:507] Master 127.15.205.62:41741 requested a full tablet report, sending...
I20251028 21:11:25.659803 19458 ts_manager.cc:194] Registered new tserver with Master: ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:25.660100 19458 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55018
I20251028 21:11:25.660477 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003279539s
I20251028 21:11:26.620308 19587 heartbeater.cc:499] Master 127.15.205.62:41741 was elected leader, sending a full tablet report...
I20251028 21:11:26.634645 19663 heartbeater.cc:499] Master 127.15.205.62:41741 was elected leader, sending a full tablet report...
I20251028 21:11:26.660497 19738 heartbeater.cc:499] Master 127.15.205.62:41741 was elected leader, sending a full tablet report...
I20251028 21:11:26.685514 16180 test_util.cc:276] Using random seed: 1971016009
I20251028 21:11:26.689870 19458 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55020:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu""\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:11:26.690025 19458 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:11:26.695246 19545 tablet_service.cc:1505] Processing CreateTablet for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251028 21:11:26.695325 19551 tablet_service.cc:1505] Processing CreateTablet for tablet ff26670b4bcc4167867fa54b97417365 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251028 21:11:26.695398 19545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49b5a0ce6fe4d91aa0ab60a1d770949. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.695431 19550 tablet_service.cc:1505] Processing CreateTablet for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251028 21:11:26.695537 19550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.695741 19622 tablet_service.cc:1505] Processing CreateTablet for tablet 352c91b410644a06914dd52b1ba5c584 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251028 21:11:26.695849 19622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 352c91b410644a06914dd52b1ba5c584. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.697537 19548 tablet_service.cc:1505] Processing CreateTablet for tablet 10b6448e96bb4fd08006249ba36d650e (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251028 21:11:26.697624 19626 tablet_service.cc:1505] Processing CreateTablet for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251028 21:11:26.697781 19626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.698661 19625 tablet_service.cc:1505] Processing CreateTablet for tablet 590f65c9d19a4688b25734528ac7474f (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251028 21:11:26.698750 19625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590f65c9d19a4688b25734528ac7474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.698896 19626 tablet_service.cc:1505] Processing CreateTablet for tablet e83bd2a0bd3c40c1b6abb84a48fa9013 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251028 21:11:26.698916 19622 tablet_service.cc:1505] Processing CreateTablet for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251028 21:11:26.698958 19626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83bd2a0bd3c40c1b6abb84a48fa9013. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.698982 19547 tablet_service.cc:1505] Processing CreateTablet for tablet 4c67b4b90e724c8492a4c82e84487f23 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251028 21:11:26.699007 19622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.699126 19623 tablet_service.cc:1505] Processing CreateTablet for tablet 4c67b4b90e724c8492a4c82e84487f23 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251028 21:11:26.699188 19623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c67b4b90e724c8492a4c82e84487f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.700023 19624 tablet_service.cc:1505] Processing CreateTablet for tablet 10b6448e96bb4fd08006249ba36d650e (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251028 21:11:26.700104 19624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b6448e96bb4fd08006249ba36d650e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.697623 19627 tablet_service.cc:1505] Processing CreateTablet for tablet ff26670b4bcc4167867fa54b97417365 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251028 21:11:26.700227 19627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff26670b4bcc4167867fa54b97417365. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.700471 19621 tablet_service.cc:1505] Processing CreateTablet for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251028 21:11:26.700548 19621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49b5a0ce6fe4d91aa0ab60a1d770949. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.701105 19620 tablet_service.cc:1505] Processing CreateTablet for tablet b22601bb9db44a3489691047e5a7c40a (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251028 21:11:26.701177 19620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b22601bb9db44a3489691047e5a7c40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.701587 19542 tablet_service.cc:1505] Processing CreateTablet for tablet 903511324c64491b827a522851cb2402 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251028 21:11:26.701686 19619 tablet_service.cc:1505] Processing CreateTablet for tablet e684839cc93d4e4685039496fbd01e03 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251028 21:11:26.701730 19543 tablet_service.cc:1505] Processing CreateTablet for tablet e684839cc93d4e4685039496fbd01e03 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251028 21:11:26.701764 19619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e684839cc93d4e4685039496fbd01e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.702200 19545 tablet_service.cc:1505] Processing CreateTablet for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251028 21:11:26.702237 19618 tablet_service.cc:1505] Processing CreateTablet for tablet 903511324c64491b827a522851cb2402 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251028 21:11:26.698928 19550 tablet_service.cc:1505] Processing CreateTablet for tablet e83bd2a0bd3c40c1b6abb84a48fa9013 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251028 21:11:26.702298 19544 tablet_service.cc:1505] Processing CreateTablet for tablet b22601bb9db44a3489691047e5a7c40a (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251028 21:11:26.702332 19618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903511324c64491b827a522851cb2402. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.702250 19699 tablet_service.cc:1505] Processing CreateTablet for tablet 10b6448e96bb4fd08006249ba36d650e (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251028 21:11:26.702440 19699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b6448e96bb4fd08006249ba36d650e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.702780 19695 tablet_service.cc:1505] Processing CreateTablet for tablet b22601bb9db44a3489691047e5a7c40a (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251028 21:11:26.702821 19697 tablet_service.cc:1505] Processing CreateTablet for tablet 352c91b410644a06914dd52b1ba5c584 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251028 21:11:26.702860 19695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b22601bb9db44a3489691047e5a7c40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.702898 19697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 352c91b410644a06914dd52b1ba5c584. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.702981 19546 tablet_service.cc:1505] Processing CreateTablet for tablet 352c91b410644a06914dd52b1ba5c584 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251028 21:11:26.703111 19694 tablet_service.cc:1505] Processing CreateTablet for tablet e684839cc93d4e4685039496fbd01e03 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251028 21:11:26.703187 19694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e684839cc93d4e4685039496fbd01e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.703760 19699 tablet_service.cc:1505] Processing CreateTablet for tablet e83bd2a0bd3c40c1b6abb84a48fa9013 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251028 21:11:26.703841 19699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83bd2a0bd3c40c1b6abb84a48fa9013. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.703979 19697 tablet_service.cc:1505] Processing CreateTablet for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251028 21:11:26.704033 19697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.704361 19701 tablet_service.cc:1505] Processing CreateTablet for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251028 21:11:26.704435 19701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.702430 19758 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.705367 19758 tablet_bootstrap.cc:654] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.705621 19698 tablet_service.cc:1505] Processing CreateTablet for tablet 4c67b4b90e724c8492a4c82e84487f23 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251028 21:11:26.705715 19698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c67b4b90e724c8492a4c82e84487f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.705788 19760 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.706239 19760 tablet_bootstrap.cc:654] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.706620 19758 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.706655 19758 ts_tablet_manager.cc:1403] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251028 21:11:26.706787 19758 raft_consensus.cc:359] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.706844 19758 raft_consensus.cc:385] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.706864 19758 raft_consensus.cc:740] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.706925 19758 consensus_queue.cc:260] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.706946 19702 tablet_service.cc:1505] Processing CreateTablet for tablet ff26670b4bcc4167867fa54b97417365 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251028 21:11:26.707042 19700 tablet_service.cc:1505] Processing CreateTablet for tablet 590f65c9d19a4688b25734528ac7474f (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251028 21:11:26.707082 19702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff26670b4bcc4167867fa54b97417365. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.707198 19696 tablet_service.cc:1505] Processing CreateTablet for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251028 21:11:26.707258 19758 ts_tablet_manager.cc:1434] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.707473 19758 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.697635 19548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b6448e96bb4fd08006249ba36d650e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.707731 19544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b22601bb9db44a3489691047e5a7c40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.707839 19758 tablet_bootstrap.cc:654] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.708020 19542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903511324c64491b827a522851cb2402. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.708316 19760 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.708354 19760 ts_tablet_manager.cc:1403] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:11:26.708380 19550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83bd2a0bd3c40c1b6abb84a48fa9013. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.708478 19760 raft_consensus.cc:359] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.708524 19760 raft_consensus.cc:385] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.708540 19760 raft_consensus.cc:740] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.708595 19760 consensus_queue.cc:260] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.708729 19760 ts_tablet_manager.cc:1434] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.708926 19551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff26670b4bcc4167867fa54b97417365. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.709113 19760 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.709345 19543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e684839cc93d4e4685039496fbd01e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.707361 19693 tablet_service.cc:1505] Processing CreateTablet for tablet 903511324c64491b827a522851cb2402 (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251028 21:11:26.707402 19549 tablet_service.cc:1505] Processing CreateTablet for tablet 590f65c9d19a4688b25734528ac7474f (DEFAULT_TABLE table=test-workload [id=19436255528a4488ac9e29c7f47516fb]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251028 21:11:26.710327 19549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590f65c9d19a4688b25734528ac7474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.710477 19545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.711349 19547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c67b4b90e724c8492a4c82e84487f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.712064 19546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 352c91b410644a06914dd52b1ba5c584. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.707294 19696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49b5a0ce6fe4d91aa0ab60a1d770949. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.713114 19700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590f65c9d19a4688b25734528ac7474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.713194 19693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903511324c64491b827a522851cb2402. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:26.714735 19760 tablet_bootstrap.cc:654] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.714987 19759 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.715184 19760 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.715224 19760 ts_tablet_manager.cc:1403] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20251028 21:11:26.715330 19759 tablet_bootstrap.cc:654] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.715390 19760 raft_consensus.cc:359] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.715461 19760 raft_consensus.cc:385] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.715487 19760 raft_consensus.cc:740] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.715531 19760 consensus_queue.cc:260] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.715576 19758 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.715610 19758 ts_tablet_manager.cc:1403] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20251028 21:11:26.715615 19760 ts_tablet_manager.cc:1434] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.715785 19758 raft_consensus.cc:359] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.715842 19758 raft_consensus.cc:385] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.715866 19758 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.715905 19758 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.715981 19758 ts_tablet_manager.cc:1434] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.715812 19760 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.716087 19759 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.716159 19758 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.716180 19759 ts_tablet_manager.cc:1403] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.716326 19759 raft_consensus.cc:359] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.716363 19760 tablet_bootstrap.cc:654] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.716382 19759 raft_consensus.cc:385] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.716403 19759 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.716455 19759 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.716495 19758 tablet_bootstrap.cc:654] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.716600 19759 ts_tablet_manager.cc:1434] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.716720 19759 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.717077 19759 tablet_bootstrap.cc:654] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.717079 19758 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.717139 19758 ts_tablet_manager.cc:1403] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.717248 19758 raft_consensus.cc:359] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.717311 19758 raft_consensus.cc:385] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.717334 19758 raft_consensus.cc:740] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.717379 19758 consensus_queue.cc:260] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.717449 19758 ts_tablet_manager.cc:1434] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.717521 19758 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.717674 19760 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.717715 19760 ts_tablet_manager.cc:1403] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:11:26.717868 19758 tablet_bootstrap.cc:654] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.717837 19760 raft_consensus.cc:359] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.717922 19760 raft_consensus.cc:385] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.717950 19760 raft_consensus.cc:740] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.718003 19759 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.717993 19760 consensus_queue.cc:260] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.718030 19759 ts_tablet_manager.cc:1403] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.718071 19760 ts_tablet_manager.cc:1434] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.718185 19759 raft_consensus.cc:359] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.718384 19759 raft_consensus.cc:385] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.718401 19759 raft_consensus.cc:740] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.718443 19759 consensus_queue.cc:260] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.718523 19759 ts_tablet_manager.cc:1434] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.718572 19759 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.718358 19760 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.718977 19759 tablet_bootstrap.cc:654] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.718997 19760 tablet_bootstrap.cc:654] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.718406 19758 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.719435 19758 ts_tablet_manager.cc:1403] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:11:26.719555 19758 raft_consensus.cc:359] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.719616 19758 raft_consensus.cc:385] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.719642 19758 raft_consensus.cc:740] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.719693 19760 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.719738 19760 ts_tablet_manager.cc:1403] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:11:26.719563 19759 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.719703 19758 consensus_queue.cc:260] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.719823 19759 ts_tablet_manager.cc:1403] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.719907 19758 ts_tablet_manager.cc:1434] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.719928 19760 raft_consensus.cc:359] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.719964 19758 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.719975 19760 raft_consensus.cc:385] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.719990 19759 raft_consensus.cc:359] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.719996 19760 raft_consensus.cc:740] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.720043 19759 raft_consensus.cc:385] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.720058 19759 raft_consensus.cc:740] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.720077 19760 consensus_queue.cc:260] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.720095 19759 consensus_queue.cc:260] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.720165 19760 ts_tablet_manager.cc:1434] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.720219 19760 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.720240 19759 ts_tablet_manager.cc:1434] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.720301 19759 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.720538 19758 tablet_bootstrap.cc:654] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.720628 19760 tablet_bootstrap.cc:654] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.721153 19758 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.721192 19758 ts_tablet_manager.cc:1403] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.720660 19759 tablet_bootstrap.cc:654] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.721326 19758 raft_consensus.cc:359] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.721376 19758 raft_consensus.cc:385] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.721395 19758 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.721433 19758 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.721506 19760 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.721544 19760 ts_tablet_manager.cc:1403] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.721663 19760 raft_consensus.cc:359] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.721506 19758 ts_tablet_manager.cc:1434] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.721755 19760 raft_consensus.cc:385] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.721782 19760 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.721803 19758 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.721834 19760 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.721994 19760 ts_tablet_manager.cc:1434] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.722043 19760 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.722152 19758 tablet_bootstrap.cc:654] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.722242 19759 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.722277 19759 ts_tablet_manager.cc:1403] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:11:26.722492 19760 tablet_bootstrap.cc:654] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.722486 19759 raft_consensus.cc:359] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.722568 19759 raft_consensus.cc:385] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.722615 19759 raft_consensus.cc:740] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.722661 19759 consensus_queue.cc:260] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.722723 19758 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.722754 19759 ts_tablet_manager.cc:1434] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.722762 19758 ts_tablet_manager.cc:1403] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.722929 19758 raft_consensus.cc:359] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.722971 19758 raft_consensus.cc:385] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.722992 19758 raft_consensus.cc:740] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.722997 19760 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.723042 19758 consensus_queue.cc:260] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.723059 19760 ts_tablet_manager.cc:1403] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.723124 19758 ts_tablet_manager.cc:1434] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.723181 19758 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.723201 19760 raft_consensus.cc:359] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.723306 19760 raft_consensus.cc:385] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.723330 19760 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.723392 19760 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.723567 19760 ts_tablet_manager.cc:1434] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.723618 19758 tablet_bootstrap.cc:654] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.723650 19760 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.724050 19760 tablet_bootstrap.cc:654] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.724073 19759 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.724467 19759 tablet_bootstrap.cc:654] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.724558 19760 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.724588 19760 ts_tablet_manager.cc:1403] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.724691 19760 raft_consensus.cc:359] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.724731 19760 raft_consensus.cc:385] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.724741 19760 raft_consensus.cc:740] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.724771 19760 consensus_queue.cc:260] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.724876 19760 ts_tablet_manager.cc:1434] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.724982 19760 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.725064 19758 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.725111 19758 ts_tablet_manager.cc:1403] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:11:26.725224 19758 raft_consensus.cc:359] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.725275 19758 raft_consensus.cc:385] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.725296 19758 raft_consensus.cc:740] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.725344 19759 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.725332 19758 consensus_queue.cc:260] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.725358 19760 tablet_bootstrap.cc:654] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.725376 19759 ts_tablet_manager.cc:1403] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.725400 19758 ts_tablet_manager.cc:1434] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.725457 19758 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.725500 19759 raft_consensus.cc:359] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.725541 19759 raft_consensus.cc:385] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.725561 19759 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.725600 19759 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.725664 19759 ts_tablet_manager.cc:1434] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.725713 19759 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.725804 19758 tablet_bootstrap.cc:654] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.725831 19760 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.725924 19760 ts_tablet_manager.cc:1403] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.726020 19760 raft_consensus.cc:359] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.726052 19760 raft_consensus.cc:385] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.726064 19760 raft_consensus.cc:740] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.726096 19760 consensus_queue.cc:260] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.726117 19759 tablet_bootstrap.cc:654] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.726177 19760 ts_tablet_manager.cc:1434] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.726236 19760 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.726614 19760 tablet_bootstrap.cc:654] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.726825 19759 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.726861 19759 ts_tablet_manager.cc:1403] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.726998 19759 raft_consensus.cc:359] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.727062 19759 raft_consensus.cc:385] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.727082 19759 raft_consensus.cc:740] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.727133 19759 consensus_queue.cc:260] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.727169 19760 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.727205 19759 ts_tablet_manager.cc:1434] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.727213 19760 ts_tablet_manager.cc:1403] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.727259 19759 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.727391 19760 raft_consensus.cc:359] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.727451 19760 raft_consensus.cc:385] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.727473 19760 raft_consensus.cc:740] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.727507 19758 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.727545 19758 ts_tablet_manager.cc:1403] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:11:26.727512 19760 consensus_queue.cc:260] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.727612 19760 ts_tablet_manager.cc:1434] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.727656 19760 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.727667 19758 raft_consensus.cc:359] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.727679 19759 tablet_bootstrap.cc:654] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.727705 19758 raft_consensus.cc:385] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.727727 19758 raft_consensus.cc:740] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.727766 19758 consensus_queue.cc:260] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.727841 19758 ts_tablet_manager.cc:1434] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.727887 19758 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.728026 19760 tablet_bootstrap.cc:654] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.728303 19759 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.728274 19758 tablet_bootstrap.cc:654] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.728370 19759 ts_tablet_manager.cc:1403] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.728497 19759 raft_consensus.cc:359] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.728549 19759 raft_consensus.cc:385] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.728570 19759 raft_consensus.cc:740] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.728622 19759 consensus_queue.cc:260] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.728694 19759 ts_tablet_manager.cc:1434] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.728739 19759 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.729091 19759 tablet_bootstrap.cc:654] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.729584 19759 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.729626 19759 ts_tablet_manager.cc:1403] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.729763 19759 raft_consensus.cc:359] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.729804 19759 raft_consensus.cc:385] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.729827 19759 raft_consensus.cc:740] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.729890 19759 consensus_queue.cc:260] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.729969 19759 ts_tablet_manager.cc:1434] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.730023 19759 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.730392 19759 tablet_bootstrap.cc:654] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.730918 19759 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.730918 19760 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.730991 19759 ts_tablet_manager.cc:1403] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.730993 19760 ts_tablet_manager.cc:1403] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:11:26.731153 19760 raft_consensus.cc:359] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.731166 19759 raft_consensus.cc:359] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.731217 19760 raft_consensus.cc:385] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.731225 19759 raft_consensus.cc:385] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.731254 19760 raft_consensus.cc:740] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.731282 19759 raft_consensus.cc:740] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.731375 19759 consensus_queue.cc:260] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.731374 19760 consensus_queue.cc:260] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.731470 19760 ts_tablet_manager.cc:1434] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.731470 19759 ts_tablet_manager.cc:1434] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:11:26.731564 19759 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.731916 19759 tablet_bootstrap.cc:654] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.731526 19760 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.732445 19759 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.732479 19759 ts_tablet_manager.cc:1403] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.732525 19760 tablet_bootstrap.cc:654] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.732605 19759 raft_consensus.cc:359] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.732682 19759 raft_consensus.cc:385] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.732702 19759 raft_consensus.cc:740] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.732751 19759 consensus_queue.cc:260] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.732848 19759 ts_tablet_manager.cc:1434] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.732896 19759 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.732976 19760 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.733013 19760 ts_tablet_manager.cc:1403] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.733100 19758 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.733129 19760 raft_consensus.cc:359] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.733204 19760 raft_consensus.cc:385] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.733227 19760 raft_consensus.cc:740] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.733239 19759 tablet_bootstrap.cc:654] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.733258 19760 consensus_queue.cc:260] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.733137 19758 ts_tablet_manager.cc:1403] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20251028 21:11:26.733367 19760 ts_tablet_manager.cc:1434] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.733430 19760 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1: Bootstrap starting.
I20251028 21:11:26.733414 19758 raft_consensus.cc:359] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.733469 19758 raft_consensus.cc:385] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.733491 19758 raft_consensus.cc:740] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.733536 19758 consensus_queue.cc:260] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.733611 19758 ts_tablet_manager.cc:1434] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.733736 19758 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.733781 19760 tablet_bootstrap.cc:654] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.734120 19758 tablet_bootstrap.cc:654] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.734170 19763 raft_consensus.cc:493] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.734216 19763 raft_consensus.cc:515] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.734357 19760 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1: No bootstrap required, opened a new log
I20251028 21:11:26.734388 19763 leader_election.cc:290] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.734577 19759 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.734618 19759 ts_tablet_manager.cc:1403] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:11:26.734747 19759 raft_consensus.cc:359] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.734787 19759 raft_consensus.cc:385] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.734802 19759 raft_consensus.cc:740] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.734838 19759 consensus_queue.cc:260] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.734395 19760 ts_tablet_manager.cc:1403] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.735102 19760 raft_consensus.cc:359] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.735183 19760 raft_consensus.cc:385] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.735213 19760 raft_consensus.cc:740] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.735253 19760 consensus_queue.cc:260] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.734941 19759 ts_tablet_manager.cc:1434] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.735435 19760 ts_tablet_manager.cc:1434] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.735463 19759 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0: Bootstrap starting.
I20251028 21:11:26.736021 19759 tablet_bootstrap.cc:654] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.736506 19759 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0: No bootstrap required, opened a new log
I20251028 21:11:26.736550 19759 ts_tablet_manager.cc:1403] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:26.736687 19758 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.736685 19759 raft_consensus.cc:359] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.736752 19759 raft_consensus.cc:385] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.736775 19759 raft_consensus.cc:740] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.736819 19759 consensus_queue.cc:260] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.736932 19759 ts_tablet_manager.cc:1434] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.737004 19758 ts_tablet_manager.cc:1403] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20251028 21:11:26.737138 19758 raft_consensus.cc:359] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.737175 19758 raft_consensus.cc:385] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.737186 19758 raft_consensus.cc:740] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.737221 19758 consensus_queue.cc:260] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.737293 19758 ts_tablet_manager.cc:1434] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.737330 19758 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.737649 19758 tablet_bootstrap.cc:654] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.737643 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" is_pre_election: true
I20251028 21:11:26.737725 19712 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 0.
I20251028 21:11:26.737859 19600 leader_election.cc:304] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.737926 19763 raft_consensus.cc:2804] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.737987 19763 raft_consensus.cc:493] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.738013 19763 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.738629 19763 raft_consensus.cc:515] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.738729 19763 leader_election.cc:290] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.738852 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:26.738906 19712 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.738931 19758 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.738961 19758 ts_tablet_manager.cc:1403] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:11:26.739073 19758 raft_consensus.cc:359] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.739109 19758 raft_consensus.cc:385] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.739125 19758 raft_consensus.cc:740] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.739164 19758 consensus_queue.cc:260] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.739226 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0" is_pre_election: true
I20251028 21:11:26.739248 19560 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:26.739328 19561 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 0.
I20251028 21:11:26.739229 19758 ts_tablet_manager.cc:1434] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.739524 19758 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Bootstrap starting.
I20251028 21:11:26.739543 19712 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 1.
I20251028 21:11:26.739686 19600 leader_election.cc:304] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters: 95350159593b4057ae155343f06246d0
I20251028 21:11:26.739830 19763 raft_consensus.cc:2804] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.739918 19758 tablet_bootstrap.cc:654] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Neither blocks nor log segments found. Creating new log.
I20251028 21:11:26.739948 19763 raft_consensus.cc:697] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:26.740061 19763 consensus_queue.cc:237] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.740478 19458 catalog_manager.cc:5649] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 1 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:26.740684 19758 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: No bootstrap required, opened a new log
I20251028 21:11:26.740729 19758 ts_tablet_manager.cc:1403] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:11:26.740911 19758 raft_consensus.cc:359] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.741024 19758 raft_consensus.cc:385] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:11:26.741137 19758 raft_consensus.cc:740] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Initialized, Role: FOLLOWER
I20251028 21:11:26.741181 19758 consensus_queue.cc:260] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.741261 19758 ts_tablet_manager.cc:1434] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:26.749620 19775 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.749688 19775 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.749800 19775 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.752190 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" is_pre_election: true
I20251028 21:11:26.752241 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:26.752255 19712 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:26.752353 19637 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:26.752570 19524 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.752638 19775 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.752686 19775 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.752722 19775 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.753350 19775 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.753456 19775 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.753588 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:26.753588 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:26.753672 19712 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.753679 19637 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.754259 19712 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:26.754259 19637 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:26.754424 19527 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.754516 19775 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.754560 19775 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:26.754609 19775 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.755029 19458 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:26.758878 19765 raft_consensus.cc:493] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.758927 19765 raft_consensus.cc:515] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.759064 19765 leader_election.cc:290] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:26.761268 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f49b5a0ce6fe4d91aa0ab60a1d770949" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0" is_pre_election: true
I20251028 21:11:26.761276 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f49b5a0ce6fe4d91aa0ab60a1d770949" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:26.761354 19561 raft_consensus.cc:2468] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 0.
I20251028 21:11:26.761360 19637 raft_consensus.cc:2468] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 0.
I20251028 21:11:26.761499 19676 leader_election.cc:304] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.761592 19765 raft_consensus.cc:2804] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.761631 19765 raft_consensus.cc:493] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.761643 19765 raft_consensus.cc:3060] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.762046 19765 raft_consensus.cc:515] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.762135 19765 leader_election.cc:290] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:26.762255 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f49b5a0ce6fe4d91aa0ab60a1d770949" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:26.762264 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f49b5a0ce6fe4d91aa0ab60a1d770949" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:26.762331 19637 raft_consensus.cc:3060] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.762351 19561 raft_consensus.cc:3060] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.762876 19637 raft_consensus.cc:2468] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 1.
I20251028 21:11:26.762934 19561 raft_consensus.cc:2468] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 1.
I20251028 21:11:26.763056 19678 leader_election.cc:304] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.763144 19765 raft_consensus.cc:2804] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.763259 19765 raft_consensus.cc:697] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:26.763331 19765 consensus_queue.cc:237] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.763738 19458 catalog_manager.cc:5649] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } }
I20251028 21:11:26.767803 19775 raft_consensus.cc:493] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.767845 19775 raft_consensus.cc:515] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.767956 19775 leader_election.cc:290] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.768069 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c67b4b90e724c8492a4c82e84487f23" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" is_pre_election: true
I20251028 21:11:26.768079 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c67b4b90e724c8492a4c82e84487f23" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:26.768167 19637 raft_consensus.cc:2468] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:26.768169 19712 raft_consensus.cc:2468] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:26.768283 19527 leader_election.cc:304] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:26.768392 19775 raft_consensus.cc:2804] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.768425 19775 raft_consensus.cc:493] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.768438 19775 raft_consensus.cc:3060] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.768805 19775 raft_consensus.cc:515] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.768903 19775 leader_election.cc:290] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.769009 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c67b4b90e724c8492a4c82e84487f23" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:26.769045 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c67b4b90e724c8492a4c82e84487f23" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:26.769088 19712 raft_consensus.cc:3060] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.769093 19637 raft_consensus.cc:3060] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.769491 19712 raft_consensus.cc:2468] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:26.769528 19637 raft_consensus.cc:2468] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:26.769640 19524 leader_election.cc:304] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.769742 19775 raft_consensus.cc:2804] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.769791 19775 raft_consensus.cc:697] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:26.769835 19775 consensus_queue.cc:237] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.770251 19458 catalog_manager.cc:5649] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:26.776533 19765 raft_consensus.cc:493] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.776584 19765 raft_consensus.cc:515] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.776693 19765 leader_election.cc:290] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:26.776839 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0" is_pre_election: true
I20251028 21:11:26.776855 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:26.776891 19561 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 0.
I20251028 21:11:26.776906 19637 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 0.
I20251028 21:11:26.777019 19676 leader_election.cc:304] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.777091 19765 raft_consensus.cc:2804] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.777158 19765 raft_consensus.cc:493] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.777184 19765 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.777632 19765 raft_consensus.cc:515] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.777740 19765 leader_election.cc:290] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:26.777841 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:26.777848 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:26.777933 19637 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.777938 19561 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.778496 19637 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 1.
I20251028 21:11:26.778512 19561 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 1.
I20251028 21:11:26.778676 19678 leader_election.cc:304] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.778771 19765 raft_consensus.cc:2804] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.778815 19765 raft_consensus.cc:697] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:26.778862 19765 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.779335 19458 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } }
I20251028 21:11:26.797883 19765 raft_consensus.cc:493] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.797924 19765 raft_consensus.cc:515] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.797997 19765 leader_election.cc:290] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:26.798162 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e684839cc93d4e4685039496fbd01e03" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0" is_pre_election: true
I20251028 21:11:26.798166 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e684839cc93d4e4685039496fbd01e03" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:26.798223 19561 raft_consensus.cc:2468] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 0.
I20251028 21:11:26.798230 19637 raft_consensus.cc:2468] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 0.
I20251028 21:11:26.798357 19676 leader_election.cc:304] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.798431 19765 raft_consensus.cc:2804] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.798458 19765 raft_consensus.cc:493] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.798470 19765 raft_consensus.cc:3060] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.798808 19765 raft_consensus.cc:515] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.798908 19765 leader_election.cc:290] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:26.799026 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e684839cc93d4e4685039496fbd01e03" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:26.799036 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e684839cc93d4e4685039496fbd01e03" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:26.799082 19561 raft_consensus.cc:3060] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.799090 19637 raft_consensus.cc:3060] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.799535 19561 raft_consensus.cc:2468] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 1.
I20251028 21:11:26.799548 19637 raft_consensus.cc:2468] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 1.
I20251028 21:11:26.799708 19678 leader_election.cc:304] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.799799 19765 raft_consensus.cc:2804] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.799849 19765 raft_consensus.cc:697] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:26.799885 19765 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.800347 19458 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } }
I20251028 21:11:26.846913 19763 raft_consensus.cc:493] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.846973 19763 raft_consensus.cc:515] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.847092 19763 leader_election.cc:290] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.847246 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "10b6448e96bb4fd08006249ba36d650e" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0" is_pre_election: true
I20251028 21:11:26.847285 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "10b6448e96bb4fd08006249ba36d650e" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" is_pre_election: true
I20251028 21:11:26.847376 19561 raft_consensus.cc:2468] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 0.
I20251028 21:11:26.847384 19712 raft_consensus.cc:2468] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 0.
I20251028 21:11:26.847515 19600 leader_election.cc:304] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.847607 19763 raft_consensus.cc:2804] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.847651 19763 raft_consensus.cc:493] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.847674 19763 raft_consensus.cc:3060] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.848217 19763 raft_consensus.cc:515] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.848325 19763 leader_election.cc:290] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.848425 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "10b6448e96bb4fd08006249ba36d650e" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:26.848443 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "10b6448e96bb4fd08006249ba36d650e" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:26.848500 19561 raft_consensus.cc:3060] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.848508 19712 raft_consensus.cc:3060] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.849036 19561 raft_consensus.cc:2468] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 1.
I20251028 21:11:26.849041 19712 raft_consensus.cc:2468] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 1.
I20251028 21:11:26.849218 19600 leader_election.cc:304] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.849303 19763 raft_consensus.cc:2804] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.849362 19763 raft_consensus.cc:697] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:26.849397 19763 consensus_queue.cc:237] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.849889 19458 catalog_manager.cc:5649] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 1 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:26.856287 19765 raft_consensus.cc:493] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.856338 19765 raft_consensus.cc:515] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.856472 19765 leader_election.cc:290] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:26.856649 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:26.856703 19637 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 0.
I20251028 21:11:26.856652 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0" is_pre_election: true
I20251028 21:11:26.856743 19561 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 0.
I20251028 21:11:26.856829 19678 leader_election.cc:304] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.856936 19765 raft_consensus.cc:2804] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.856973 19765 raft_consensus.cc:493] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.856997 19765 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.857393 19765 raft_consensus.cc:515] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.857501 19765 leader_election.cc:290] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:26.857636 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:26.857692 19637 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.857707 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:26.857774 19561 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.858160 19637 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 1.
I20251028 21:11:26.858197 19561 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 1.
I20251028 21:11:26.858318 19678 leader_election.cc:304] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.858418 19765 raft_consensus.cc:2804] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.858461 19765 raft_consensus.cc:697] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:26.858506 19765 consensus_queue.cc:237] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.858954 19458 catalog_manager.cc:5649] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } }
I20251028 21:11:26.916373 19775 raft_consensus.cc:493] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.916429 19775 raft_consensus.cc:515] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.916527 19775 leader_election.cc:290] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.916682 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8798d7cdf7864b0bbd9e5b2e26d0dc4f" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:26.916731 19637 raft_consensus.cc:2468] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:26.916718 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8798d7cdf7864b0bbd9e5b2e26d0dc4f" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" is_pre_election: true
I20251028 21:11:26.916827 19712 raft_consensus.cc:2468] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:26.916886 19527 leader_election.cc:304] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:26.916965 19775 raft_consensus.cc:2804] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.917001 19775 raft_consensus.cc:493] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.917024 19775 raft_consensus.cc:3060] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.917479 19775 raft_consensus.cc:515] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.917578 19775 leader_election.cc:290] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.917695 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8798d7cdf7864b0bbd9e5b2e26d0dc4f" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:26.917702 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8798d7cdf7864b0bbd9e5b2e26d0dc4f" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:26.917774 19637 raft_consensus.cc:3060] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.917776 19712 raft_consensus.cc:3060] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.918195 19712 raft_consensus.cc:2468] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:26.918200 19637 raft_consensus.cc:2468] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:26.918347 19527 leader_election.cc:304] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:26.918424 19775 raft_consensus.cc:2804] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.918478 19775 raft_consensus.cc:697] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:26.918516 19775 consensus_queue.cc:237] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.919004 19458 catalog_manager.cc:5649] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:26.943737 19763 raft_consensus.cc:493] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:26.943790 19763 raft_consensus.cc:515] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.943917 19763 leader_election.cc:290] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.944053 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff26670b4bcc4167867fa54b97417365" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0" is_pre_election: true
I20251028 21:11:26.944064 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff26670b4bcc4167867fa54b97417365" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" is_pre_election: true
I20251028 21:11:26.944120 19561 raft_consensus.cc:2468] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 0.
I20251028 21:11:26.944144 19712 raft_consensus.cc:2468] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 0.
I20251028 21:11:26.944263 19601 leader_election.cc:304] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:26.944370 19763 raft_consensus.cc:2804] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:26.944418 19763 raft_consensus.cc:493] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:26.944437 19763 raft_consensus.cc:3060] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.944922 19763 raft_consensus.cc:515] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.945022 19763 leader_election.cc:290] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:26.945111 19561 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff26670b4bcc4167867fa54b97417365" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:26.945168 19561 raft_consensus.cc:3060] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.945196 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff26670b4bcc4167867fa54b97417365" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:26.945242 19712 raft_consensus.cc:3060] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:26.945561 19561 raft_consensus.cc:2468] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 1.
I20251028 21:11:26.945650 19712 raft_consensus.cc:2468] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 1.
I20251028 21:11:26.945708 19601 leader_election.cc:304] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:26.945801 19763 raft_consensus.cc:2804] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:26.945864 19763 raft_consensus.cc:697] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:26.945915 19763 consensus_queue.cc:237] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:26.946419 19458 catalog_manager.cc:5649] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 1 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:27.063997 19775 raft_consensus.cc:493] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:27.064081 19775 raft_consensus.cc:515] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.064245 19775 leader_election.cc:290] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:27.064412 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b22601bb9db44a3489691047e5a7c40a" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" is_pre_election: true
I20251028 21:11:27.064424 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b22601bb9db44a3489691047e5a7c40a" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:27.064487 19712 raft_consensus.cc:2468] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:27.064531 19637 raft_consensus.cc:2468] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:27.064628 19524 leader_election.cc:304] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:27.064700 19775 raft_consensus.cc:2804] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:27.064738 19775 raft_consensus.cc:493] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:27.064755 19775 raft_consensus.cc:3060] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:27.065261 19775 raft_consensus.cc:515] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.065352 19775 leader_election.cc:290] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:27.065450 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b22601bb9db44a3489691047e5a7c40a" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:27.065512 19637 raft_consensus.cc:3060] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:27.065511 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b22601bb9db44a3489691047e5a7c40a" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:27.065590 19712 raft_consensus.cc:3060] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:27.066063 19712 raft_consensus.cc:2468] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:27.066078 19637 raft_consensus.cc:2468] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:27.066221 19527 leader_election.cc:304] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:27.066303 19775 raft_consensus.cc:2804] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:27.066365 19775 raft_consensus.cc:697] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:27.066416 19775 consensus_queue.cc:237] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.066951 19458 catalog_manager.cc:5649] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:27.084457 19775 raft_consensus.cc:493] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:11:27.084511 19775 raft_consensus.cc:515] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.084630 19775 leader_election.cc:290] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:27.084733 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" is_pre_election: true
I20251028 21:11:27.084743 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" is_pre_election: true
I20251028 21:11:27.084789 19637 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:27.084805 19712 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 0.
I20251028 21:11:27.084889 19527 leader_election.cc:304] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:27.084959 19775 raft_consensus.cc:2804] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:11:27.084997 19775 raft_consensus.cc:493] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:11:27.085014 19775 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:27.085417 19775 raft_consensus.cc:515] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.085511 19775 leader_election.cc:290] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:27.085610 19637 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:27.085608 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:27.085690 19637 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:27.085697 19712 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:27.086252 19637 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:27.086252 19712 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 1.
I20251028 21:11:27.086443 19527 leader_election.cc:304] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:27.086526 19775 raft_consensus.cc:2804] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:11:27.086571 19775 raft_consensus.cc:697] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:27.086612 19775 consensus_queue.cc:237] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.087143 19458 catalog_manager.cc:5649] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:27.155140 19712 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.155320 19561 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.155392 19560 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.155611 19711 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.155690 19763 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.155776 19712 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.155791 19763 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.156080 19802 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.156180 19561 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.156219 19560 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:11:27.156373 19810 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.156814 19636 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.156914 19558 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.157068 19560 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.157142 19636 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.156198 19637 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.157080 19765 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.157377 19765 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.157440 19763 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.156196 19802 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251028 21:11:27.157202 19559 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.157648 19637 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.158300 19802 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.158478 19712 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.158574 19775 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.158924 19775 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.158931 19810 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.159098 19765 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.159199 19559 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.159102 19810 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.159994 19635 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:11:27.160267 19810 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.161223 19835 mvcc.cc:204] Tried to move back new op lower bound from 7215865393787174912 to 7215865392535576576. Current Snapshot: MvccSnapshot[applied={T|T < 7215865393787174912}]
I20251028 21:11:27.162382 19810 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.164012 19849 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.164489 19709 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:11:27.165354 19632 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:11:27.165669 19712 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:11:27.165745 19632 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:11:27.166065 19712 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:11:27.166095 19632 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:11:27.166185 19768 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.166276 19849 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.166339 19768 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.166276 19851 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.166438 19768 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.166808 19900 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.166829 19712 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:11:27.167007 19768 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:11:27.167996 19848 mvcc.cc:204] Tried to move back new op lower bound from 7215865393789046784 to 7215865392087777280. Current Snapshot: MvccSnapshot[applied={T|T < 7215865393787064320}]
I20251028 21:11:27.172134 19831 mvcc.cc:204] Tried to move back new op lower bound from 7215865393829343232 to 7215865392818782208. Current Snapshot: MvccSnapshot[applied={T|T < 7215865393824342016}]
I20251028 21:11:27.253695 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:27.254881 19935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:27.254938 19933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:27.255072 16180 server_base.cc:1047] running on GCE node
W20251028 21:11:27.255184 19932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:27.255350 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:27.255385 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:27.255403 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685887255403 us; error 0 us; skew 500 ppm
I20251028 21:11:27.256039 16180 webserver.cc:492] Webserver started at http://127.15.205.4:43935/ using document root <none> and password file <none>
I20251028 21:11:27.256119 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:27.256170 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:27.256215 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:27.256466 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-3-root/instance:
uuid: "9e69ab942ba042b6b83938c67405973c"
format_stamp: "Formatted at 2025-10-28 21:11:27 on dist-test-slave-zmkx"
I20251028 21:11:27.257654 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:27.258175 19940 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.258877 16180 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20251028 21:11:27.258934 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-3-root
uuid: "9e69ab942ba042b6b83938c67405973c"
format_stamp: "Formatted at 2025-10-28 21:11:27 on dist-test-slave-zmkx"
I20251028 21:11:27.258982 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:27.270157 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:27.270310 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:27.270416 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:27.270556 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:27.270584 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.270612 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:27.270637 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.273671 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.4:42831
I20251028 21:11:27.273718 20003 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.4:42831 every 8 connection(s)
I20251028 21:11:27.273988 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:27.274900 20008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:27.275101 20009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:27.275254 20011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:27.275377 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:27.275463 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:27.275490 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:27.275516 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685887275515 us; error 0 us; skew 500 ppm
I20251028 21:11:27.276070 16180 webserver.cc:492] Webserver started at http://127.15.205.5:46227/ using document root <none> and password file <none>
I20251028 21:11:27.276145 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:27.276180 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:27.276214 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:27.276438 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-4-root/instance:
uuid: "05d39c90d87d4cca9a23f40ad31b12e0"
format_stamp: "Formatted at 2025-10-28 21:11:27 on dist-test-slave-zmkx"
I20251028 21:11:27.276958 20004 heartbeater.cc:344] Connected to a master server at 127.15.205.62:41741
I20251028 21:11:27.277012 20004 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:27.277114 20004 heartbeater.cc:507] Master 127.15.205.62:41741 requested a full tablet report, sending...
I20251028 21:11:27.277361 19450 ts_manager.cc:194] Registered new tserver with Master: 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:27.277381 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:27.277815 19450 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55030
I20251028 21:11:27.277822 20016 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.278036 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.278076 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-4-root
uuid: "05d39c90d87d4cca9a23f40ad31b12e0"
format_stamp: "Formatted at 2025-10-28 21:11:27 on dist-test-slave-zmkx"
I20251028 21:11:27.278111 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-4-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-4-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-4-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:27.290791 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:27.291004 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:27.291141 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:27.291332 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:27.291364 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.291394 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:27.291422 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.294518 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.5:44477
I20251028 21:11:27.294637 20079 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.5:44477 every 8 connection(s)
I20251028 21:11:27.294835 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:11:27.295709 20083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:27.295758 20084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:11:27.295922 20086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:11:27.296204 16180 server_base.cc:1047] running on GCE node
I20251028 21:11:27.296293 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:11:27.296324 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:11:27.296343 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685887296343 us; error 0 us; skew 500 ppm
I20251028 21:11:27.296916 16180 webserver.cc:492] Webserver started at http://127.15.205.6:46345/ using document root <none> and password file <none>
I20251028 21:11:27.296998 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:11:27.297047 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:11:27.297115 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:11:27.297320 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-5-root/instance:
uuid: "a39fb1f187c544f2b77164040db04caf"
format_stamp: "Formatted at 2025-10-28 21:11:27 on dist-test-slave-zmkx"
I20251028 21:11:27.297583 20080 heartbeater.cc:344] Connected to a master server at 127.15.205.62:41741
I20251028 21:11:27.297636 20080 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:27.297699 20080 heartbeater.cc:507] Master 127.15.205.62:41741 requested a full tablet report, sending...
I20251028 21:11:27.297883 19450 ts_manager.cc:194] Registered new tserver with Master: 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477)
I20251028 21:11:27.298123 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:27.298281 19450 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55044
I20251028 21:11:27.298599 20091 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.298702 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.298753 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-5-root
uuid: "a39fb1f187c544f2b77164040db04caf"
format_stamp: "Formatted at 2025-10-28 21:11:27 on dist-test-slave-zmkx"
I20251028 21:11:27.298795 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-5-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-5-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-5-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:11:27.308058 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:11:27.308241 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:11:27.308377 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:11:27.308535 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:11:27.308560 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.308588 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:11:27.308614 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:27.311832 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.6:45471
I20251028 21:11:27.311901 20154 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.6:45471 every 8 connection(s)
I20251028 21:11:27.314229 20155 heartbeater.cc:344] Connected to a master server at 127.15.205.62:41741
I20251028 21:11:27.314277 20155 heartbeater.cc:461] Registering TS with master...
I20251028 21:11:27.314360 20155 heartbeater.cc:507] Master 127.15.205.62:41741 requested a full tablet report, sending...
I20251028 21:11:27.314544 19450 ts_manager.cc:194] Registered new tserver with Master: a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471)
I20251028 21:11:27.314823 19450 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55048
I20251028 21:11:27.568724 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:27.571594 19559 consensus_queue.cc:237] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:27.572184 19631 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.572216 19707 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.572386 19559 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "903511324c64491b827a522851cb2402"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:27.572446 19559 raft_consensus.cc:606] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Received request to transfer leadership to TS 6b99aaed3fec4c28bf3e3dbbd676ab65
I20251028 21:11:27.572389 19877 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.572603 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:27.572666 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:27.572619 19889 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.572996 19877 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:27.573227 19707 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:27.574285 19631 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:27.574388 19449 catalog_manager.cc:5649] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index -1 to 27, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
W20251028 21:11:27.575402 19524 consensus_peers.cc:597] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 -> Peer a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471): Couldn't send request to peer a39fb1f187c544f2b77164040db04caf. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c67b4b90e724c8492a4c82e84487f23. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:27.577288 19707 consensus_queue.cc:237] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:27.577790 19559 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.577898 19631 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.578030 19925 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.578186 19914 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.578686 19908 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:27.578773 19631 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:27.579725 19450 catalog_manager.cc:5649] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:27.580125 19559 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
W20251028 21:11:27.581215 19675 consensus_peers.cc:597] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 -> Peer 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831): Couldn't send request to peer 9e69ab942ba042b6b83938c67405973c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f49b5a0ce6fe4d91aa0ab60a1d770949. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:27.583159 19631 consensus_queue.cc:237] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:27.583547 19707 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 1 index: 26. (index mismatch)
I20251028 21:11:27.583673 19559 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 1 index: 26. (index mismatch)
I20251028 21:11:27.583820 19917 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251028 21:11:27.583920 19920 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251028 21:11:27.584420 19911 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Committing config change with OpId 1.26: config changed from index -1 to 26, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:27.584508 19707 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.26: config changed from index -1 to 26, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:27.584532 19559 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.26: config changed from index -1 to 26, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:27.585376 19450 catalog_manager.cc:5649] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index -1 to 26, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New cstate: current_term: 1 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:27.585942 19559 consensus_queue.cc:237] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:27.586405 19707 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
W20251028 21:11:27.586413 19524 consensus_peers.cc:597] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 -> Peer a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471): Couldn't send request to peer a39fb1f187c544f2b77164040db04caf. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b22601bb9db44a3489691047e5a7c40a. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:27.586468 19631 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.586629 19889 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.586751 19883 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.587284 19883 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
W20251028 21:11:27.587442 19601 consensus_peers.cc:597] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 -> Peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477): Couldn't send request to peer 05d39c90d87d4cca9a23f40ad31b12e0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff26670b4bcc4167867fa54b97417365. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:27.587427 19631 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:27.588002 19707 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:27.588161 19451 catalog_manager.cc:5649] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index -1 to 27, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:27.588251 19559 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:27.588670 19631 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.588827 19707 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.588836 19890 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.588999 19889 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.589334 19890 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:27.589445 19631 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:27.589628 19707 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:27.590435 19449 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:27.591598 19712 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:27.592000 19631 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.592206 19910 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.592299 19559 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:27.592438 19910 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:27.592698 19912 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
W20251028 21:11:27.593200 19524 consensus_peers.cc:597] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 -> Peer 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831): Couldn't send request to peer 9e69ab942ba042b6b83938c67405973c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e83bd2a0bd3c40c1b6abb84a48fa9013. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:27.593653 19631 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:27.594470 19449 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index -1 to 27, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:27.594949 19559 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
W20251028 21:11:27.595687 19676 consensus_peers.cc:597] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 -> Peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477): Couldn't send request to peer 05d39c90d87d4cca9a23f40ad31b12e0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e684839cc93d4e4685039496fbd01e03. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:27.642747 19883 raft_consensus.cc:993] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: : Instructing follower 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:27.642808 19883 raft_consensus.cc:1081] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Signalling peer 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:27.642949 19631 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "903511324c64491b827a522851cb2402"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:37070
I20251028 21:11:27.643011 19631 raft_consensus.cc:493] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:27.643038 19631 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:27.643574 19631 raft_consensus.cc:515] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.643808 19631 leader_election.cc:290] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 2 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:27.643798 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:27.643940 19558 raft_consensus.cc:3055] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:27.643797 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:27.643963 19558 raft_consensus.cc:740] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:27.643985 19712 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:27.644006 19558 consensus_queue.cc:260] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.644084 19558 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:27.644418 19712 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 2.
I20251028 21:11:27.644518 19558 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 2.
I20251028 21:11:27.644582 19600 leader_election.cc:304] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:27.644650 19920 raft_consensus.cc:2804] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:27.644692 19920 raft_consensus.cc:697] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:27.644731 19920 consensus_queue.cc:237] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:27.645119 19451 catalog_manager.cc:5649] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 1 to 2, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:27.962626 20167 ts_tablet_manager.cc:933] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: Initiating tablet copy from peer 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:27.962812 20167 tablet_copy_client.cc:323] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:36457
I20251028 21:11:27.965775 19647 tablet_copy_service.cc:140] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Received BeginTabletCopySession request for tablet ff26670b4bcc4167867fa54b97417365 from peer 05d39c90d87d4cca9a23f40ad31b12e0 ({username='slave'} at 127.0.0.1:37100)
I20251028 21:11:27.965875 19647 tablet_copy_service.cc:161] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Beginning new tablet copy session on tablet ff26670b4bcc4167867fa54b97417365 from peer 05d39c90d87d4cca9a23f40ad31b12e0 at {username='slave'} at 127.0.0.1:37100: session id = 05d39c90d87d4cca9a23f40ad31b12e0-ff26670b4bcc4167867fa54b97417365
I20251028 21:11:27.965793 20170 ts_tablet_manager.cc:933] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: Initiating tablet copy from peer 95350159593b4057ae155343f06246d0 (127.15.205.1:38429)
I20251028 21:11:27.966142 20170 tablet_copy_client.cc:323] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.1:38429
I20251028 21:11:27.966524 19647 tablet_copy_source_session.cc:215] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:27.966822 20167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff26670b4bcc4167867fa54b97417365. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:27.968361 20167 tablet_copy_client.cc:806] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:27.968495 20167 tablet_copy_client.cc:670] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:27.968952 20168 tablet_copy_client.cc:1025] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:27.969178 19571 tablet_copy_service.cc:140] P 95350159593b4057ae155343f06246d0: Received BeginTabletCopySession request for tablet 4c67b4b90e724c8492a4c82e84487f23 from peer a39fb1f187c544f2b77164040db04caf ({username='slave'} at 127.0.0.1:53836)
I20251028 21:11:27.969234 19571 tablet_copy_service.cc:161] P 95350159593b4057ae155343f06246d0: Beginning new tablet copy session on tablet 4c67b4b90e724c8492a4c82e84487f23 from peer a39fb1f187c544f2b77164040db04caf at {username='slave'} at 127.0.0.1:53836: session id = a39fb1f187c544f2b77164040db04caf-4c67b4b90e724c8492a4c82e84487f23
I20251028 21:11:27.969820 19571 tablet_copy_source_session.cc:215] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:27.970077 20170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c67b4b90e724c8492a4c82e84487f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:27.971280 20170 tablet_copy_client.cc:806] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:27.971386 20170 tablet_copy_client.cc:670] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:27.971812 20171 tablet_copy_client.cc:1025] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:28.029598 20173 ts_tablet_manager.cc:933] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: Initiating tablet copy from peer 95350159593b4057ae155343f06246d0 (127.15.205.1:38429)
I20251028 21:11:28.029767 20173 tablet_copy_client.cc:323] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.1:38429
I20251028 21:11:28.032531 19571 tablet_copy_service.cc:140] P 95350159593b4057ae155343f06246d0: Received BeginTabletCopySession request for tablet e83bd2a0bd3c40c1b6abb84a48fa9013 from peer 9e69ab942ba042b6b83938c67405973c ({username='slave'} at 127.0.0.1:53846)
I20251028 21:11:28.032590 19571 tablet_copy_service.cc:161] P 95350159593b4057ae155343f06246d0: Beginning new tablet copy session on tablet e83bd2a0bd3c40c1b6abb84a48fa9013 from peer 9e69ab942ba042b6b83938c67405973c at {username='slave'} at 127.0.0.1:53846: session id = 9e69ab942ba042b6b83938c67405973c-e83bd2a0bd3c40c1b6abb84a48fa9013
I20251028 21:11:28.033111 19571 tablet_copy_source_session.cc:215] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:28.033361 20173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83bd2a0bd3c40c1b6abb84a48fa9013. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:28.036247 20173 tablet_copy_client.cc:806] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:28.036367 20173 tablet_copy_client.cc:670] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:28.036888 20174 tablet_copy_client.cc:1025] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:28.045717 20176 ts_tablet_manager.cc:933] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: Initiating tablet copy from peer 95350159593b4057ae155343f06246d0 (127.15.205.1:38429)
I20251028 21:11:28.045909 20176 tablet_copy_client.cc:323] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.1:38429
I20251028 21:11:28.046062 19571 tablet_copy_service.cc:140] P 95350159593b4057ae155343f06246d0: Received BeginTabletCopySession request for tablet b22601bb9db44a3489691047e5a7c40a from peer a39fb1f187c544f2b77164040db04caf ({username='slave'} at 127.0.0.1:53836)
I20251028 21:11:28.046105 19571 tablet_copy_service.cc:161] P 95350159593b4057ae155343f06246d0: Beginning new tablet copy session on tablet b22601bb9db44a3489691047e5a7c40a from peer a39fb1f187c544f2b77164040db04caf at {username='slave'} at 127.0.0.1:53836: session id = a39fb1f187c544f2b77164040db04caf-b22601bb9db44a3489691047e5a7c40a
I20251028 21:11:28.046517 19571 tablet_copy_source_session.cc:215] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:28.046757 20176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b22601bb9db44a3489691047e5a7c40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:28.047914 20176 tablet_copy_client.cc:806] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:28.048019 20176 tablet_copy_client.cc:670] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:28.048561 20177 tablet_copy_client.cc:1025] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:28.079978 20178 ts_tablet_manager.cc:933] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: Initiating tablet copy from peer ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:28.080123 20178 tablet_copy_client.cc:323] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:42465
I20251028 21:11:28.082888 19722 tablet_copy_service.cc:140] P ac203379ff4e4fdbac249bcd15cf5be1: Received BeginTabletCopySession request for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 from peer 9e69ab942ba042b6b83938c67405973c ({username='slave'} at 127.0.0.1:41448)
I20251028 21:11:28.082938 19722 tablet_copy_service.cc:161] P ac203379ff4e4fdbac249bcd15cf5be1: Beginning new tablet copy session on tablet f49b5a0ce6fe4d91aa0ab60a1d770949 from peer 9e69ab942ba042b6b83938c67405973c at {username='slave'} at 127.0.0.1:41448: session id = 9e69ab942ba042b6b83938c67405973c-f49b5a0ce6fe4d91aa0ab60a1d770949
I20251028 21:11:28.083521 19722 tablet_copy_source_session.cc:215] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:28.083784 20178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49b5a0ce6fe4d91aa0ab60a1d770949. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:28.084904 20178 tablet_copy_client.cc:806] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:28.085019 20178 tablet_copy_client.cc:670] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:28.085492 20179 tablet_copy_client.cc:1025] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:28.105736 20181 ts_tablet_manager.cc:933] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: Initiating tablet copy from peer ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:28.105890 20181 tablet_copy_client.cc:323] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:42465
I20251028 21:11:28.108457 19722 tablet_copy_service.cc:140] P ac203379ff4e4fdbac249bcd15cf5be1: Received BeginTabletCopySession request for tablet e684839cc93d4e4685039496fbd01e03 from peer 05d39c90d87d4cca9a23f40ad31b12e0 ({username='slave'} at 127.0.0.1:41454)
I20251028 21:11:28.108501 19722 tablet_copy_service.cc:161] P ac203379ff4e4fdbac249bcd15cf5be1: Beginning new tablet copy session on tablet e684839cc93d4e4685039496fbd01e03 from peer 05d39c90d87d4cca9a23f40ad31b12e0 at {username='slave'} at 127.0.0.1:41454: session id = 05d39c90d87d4cca9a23f40ad31b12e0-e684839cc93d4e4685039496fbd01e03
I20251028 21:11:28.108937 19722 tablet_copy_source_session.cc:215] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:28.109145 20181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e684839cc93d4e4685039496fbd01e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:28.110224 20181 tablet_copy_client.cc:806] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:28.110339 20181 tablet_copy_client.cc:670] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:28.110809 20182 tablet_copy_client.cc:1025] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:28.189075 19707 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251028 21:11:28.189275 19917 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:28.190294 19559 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251028 21:11:28.190514 20183 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:28.278329 20004 heartbeater.cc:499] Master 127.15.205.62:41741 was elected leader, sending a full tablet report...
I20251028 21:11:28.298702 20080 heartbeater.cc:499] Master 127.15.205.62:41741 was elected leader, sending a full tablet report...
I20251028 21:11:28.315214 20155 heartbeater.cc:499] Master 127.15.205.62:41741 was elected leader, sending a full tablet report...
I20251028 21:11:29.572796 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:29.573037 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:11:29.573113 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:29.969141 20167 tablet_copy_client.cc:538] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:29.970130 20167 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap starting.
I20251028 21:11:29.972000 20170 tablet_copy_client.cc:538] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:29.972718 20170 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: Bootstrap starting.
I20251028 21:11:29.974213 20167 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=0 applied=26 ignored=0} inserts{seen=102 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:29.974392 20167 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap complete.
I20251028 21:11:29.974493 20167 ts_tablet_manager.cc:1403] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent bootstrapping tablet: real 0.004s user 0.005s sys 0.000s
I20251028 21:11:29.974642 20167 raft_consensus.cc:359] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:29.974730 20167 raft_consensus.cc:740] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 05d39c90d87d4cca9a23f40ad31b12e0, State: Initialized, Role: LEARNER
I20251028 21:11:29.974766 20167 consensus_queue.cc:260] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 26 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:29.974910 20167 ts_tablet_manager.cc:1434] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:11:29.975265 19647 tablet_copy_service.cc:342] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Request end of tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-ff26670b4bcc4167867fa54b97417365 received from {username='slave'} at 127.0.0.1:37100
I20251028 21:11:29.975337 19647 tablet_copy_service.cc:434] P 6b99aaed3fec4c28bf3e3dbbd676ab65: ending tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-ff26670b4bcc4167867fa54b97417365 on tablet ff26670b4bcc4167867fa54b97417365 with peer 05d39c90d87d4cca9a23f40ad31b12e0
I20251028 21:11:29.977058 20170 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=110 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:29.977259 20170 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: Bootstrap complete.
I20251028 21:11:29.977356 20170 ts_tablet_manager.cc:1403] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 21:11:29.977480 20170 raft_consensus.cc:359] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:29.977530 20170 raft_consensus.cc:740] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a39fb1f187c544f2b77164040db04caf, State: Initialized, Role: LEARNER
I20251028 21:11:29.977559 20170 consensus_queue.cc:260] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:29.977670 20170 ts_tablet_manager.cc:1434] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: Time spent starting tablet: real 0.000s user 0.000s sys 0.002s
I20251028 21:11:29.977843 19571 tablet_copy_service.cc:342] P 95350159593b4057ae155343f06246d0: Request end of tablet copy session a39fb1f187c544f2b77164040db04caf-4c67b4b90e724c8492a4c82e84487f23 received from {username='slave'} at 127.0.0.1:53836
I20251028 21:11:29.977893 19571 tablet_copy_service.cc:434] P 95350159593b4057ae155343f06246d0: ending tablet copy session a39fb1f187c544f2b77164040db04caf-4c67b4b90e724c8492a4c82e84487f23 on tablet 4c67b4b90e724c8492a4c82e84487f23 with peer a39fb1f187c544f2b77164040db04caf
I20251028 21:11:30.037087 20173 tablet_copy_client.cc:538] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:30.037955 20173 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: Bootstrap starting.
I20251028 21:11:30.040966 20173 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=111 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:30.041153 20173 tablet_bootstrap.cc:492] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: Bootstrap complete.
I20251028 21:11:30.041263 20173 ts_tablet_manager.cc:1403] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 21:11:30.041392 20173 raft_consensus.cc:359] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:30.041471 20173 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9e69ab942ba042b6b83938c67405973c, State: Initialized, Role: LEARNER
I20251028 21:11:30.041512 20173 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:30.041627 20173 ts_tablet_manager.cc:1434] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: Time spent starting tablet: real 0.000s user 0.002s sys 0.000s
I20251028 21:11:30.041795 19571 tablet_copy_service.cc:342] P 95350159593b4057ae155343f06246d0: Request end of tablet copy session 9e69ab942ba042b6b83938c67405973c-e83bd2a0bd3c40c1b6abb84a48fa9013 received from {username='slave'} at 127.0.0.1:53846
I20251028 21:11:30.041852 19571 tablet_copy_service.cc:434] P 95350159593b4057ae155343f06246d0: ending tablet copy session 9e69ab942ba042b6b83938c67405973c-e83bd2a0bd3c40c1b6abb84a48fa9013 on tablet e83bd2a0bd3c40c1b6abb84a48fa9013 with peer 9e69ab942ba042b6b83938c67405973c
I20251028 21:11:30.048743 20176 tablet_copy_client.cc:538] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:30.049325 20176 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: Bootstrap starting.
I20251028 21:11:30.052659 20176 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=91 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:30.052832 20176 tablet_bootstrap.cc:492] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: Bootstrap complete.
I20251028 21:11:30.052948 20176 ts_tablet_manager.cc:1403] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 21:11:30.053082 20176 raft_consensus.cc:359] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:30.053154 20176 raft_consensus.cc:740] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a39fb1f187c544f2b77164040db04caf, State: Initialized, Role: LEARNER
I20251028 21:11:30.053185 20176 consensus_queue.cc:260] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:30.053292 20176 ts_tablet_manager.cc:1434] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: Time spent starting tablet: real 0.000s user 0.002s sys 0.000s
I20251028 21:11:30.053431 19571 tablet_copy_service.cc:342] P 95350159593b4057ae155343f06246d0: Request end of tablet copy session a39fb1f187c544f2b77164040db04caf-b22601bb9db44a3489691047e5a7c40a received from {username='slave'} at 127.0.0.1:53836
I20251028 21:11:30.053485 19571 tablet_copy_service.cc:434] P 95350159593b4057ae155343f06246d0: ending tablet copy session a39fb1f187c544f2b77164040db04caf-b22601bb9db44a3489691047e5a7c40a on tablet b22601bb9db44a3489691047e5a7c40a with peer a39fb1f187c544f2b77164040db04caf
I20251028 21:11:30.085660 20178 tablet_copy_client.cc:538] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:30.086503 20178 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: Bootstrap starting.
I20251028 21:11:30.090401 20178 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=114 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:30.090590 20178 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: Bootstrap complete.
I20251028 21:11:30.090688 20178 ts_tablet_manager.cc:1403] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 21:11:30.090837 20178 raft_consensus.cc:359] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:30.090916 20178 raft_consensus.cc:740] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9e69ab942ba042b6b83938c67405973c, State: Initialized, Role: LEARNER
I20251028 21:11:30.090946 20178 consensus_queue.cc:260] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:30.091041 20178 ts_tablet_manager.cc:1434] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: Time spent starting tablet: real 0.000s user 0.000s sys 0.003s
I20251028 21:11:30.091318 19722 tablet_copy_service.cc:342] P ac203379ff4e4fdbac249bcd15cf5be1: Request end of tablet copy session 9e69ab942ba042b6b83938c67405973c-f49b5a0ce6fe4d91aa0ab60a1d770949 received from {username='slave'} at 127.0.0.1:41448
I20251028 21:11:30.091364 19722 tablet_copy_service.cc:434] P ac203379ff4e4fdbac249bcd15cf5be1: ending tablet copy session 9e69ab942ba042b6b83938c67405973c-f49b5a0ce6fe4d91aa0ab60a1d770949 on tablet f49b5a0ce6fe4d91aa0ab60a1d770949 with peer 9e69ab942ba042b6b83938c67405973c
I20251028 21:11:30.110984 20181 tablet_copy_client.cc:538] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:30.111771 20181 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap starting.
I20251028 21:11:30.115501 20181 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=93 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:30.115686 20181 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap complete.
I20251028 21:11:30.115785 20181 ts_tablet_manager.cc:1403] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251028 21:11:30.115921 20181 raft_consensus.cc:359] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:30.116124 20181 raft_consensus.cc:740] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 05d39c90d87d4cca9a23f40ad31b12e0, State: Initialized, Role: LEARNER
I20251028 21:11:30.116174 20181 consensus_queue.cc:260] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:30.116379 20181 ts_tablet_manager.cc:1434] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:11:30.116526 19722 tablet_copy_service.cc:342] P ac203379ff4e4fdbac249bcd15cf5be1: Request end of tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-e684839cc93d4e4685039496fbd01e03 received from {username='slave'} at 127.0.0.1:41454
I20251028 21:11:30.116577 19722 tablet_copy_service.cc:434] P ac203379ff4e4fdbac249bcd15cf5be1: ending tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-e684839cc93d4e4685039496fbd01e03 on tablet e684839cc93d4e4685039496fbd01e03 with peer 05d39c90d87d4cca9a23f40ad31b12e0
I20251028 21:11:30.137092 20054 raft_consensus.cc:1217] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.25->[1.26-1.26] Dedup: 1.26->[]
I20251028 21:11:30.180893 20054 raft_consensus.cc:1217] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251028 21:11:30.268939 20129 raft_consensus.cc:1217] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251028 21:11:30.269891 20129 raft_consensus.cc:1217] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251028 21:11:30.349644 19978 raft_consensus.cc:1217] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251028 21:11:30.477641 19978 raft_consensus.cc:1217] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251028 21:11:30.695997 20201 raft_consensus.cc:1064] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: attempting to promote NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 to VOTER
I20251028 21:11:30.696264 20201 consensus_queue.cc:237] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:30.696795 19707 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:30.696842 19558 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:30.696977 20201 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:30.697224 20190 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:30.696806 20054 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 LEARNER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:30.697307 20204 raft_consensus.cc:1064] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: attempting to promote NON_VOTER a39fb1f187c544f2b77164040db04caf to VOTER
I20251028 21:11:30.697602 20204 consensus_queue.cc:237] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:30.697647 20190 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:30.698101 20190 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index 26 to 27, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.698210 19707 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index 26 to 27, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.699237 19451 catalog_manager.cc:5649] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 26 to 27, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.699354 19558 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index 26 to 27, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.699313 20129 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.699677 20205 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.700006 19712 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.700261 19631 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.700343 20204 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.700510 20204 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.701061 20054 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index 26 to 27, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.701406 20213 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.701614 19633 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.702391 19631 consensus_queue.cc:237] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:30.702421 20129 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.702525 19707 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.702987 19451 catalog_manager.cc:5649] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.703250 20054 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.703506 19558 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.703696 20190 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.703892 20190 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.704299 20190 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.704412 19558 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.704644 19559 consensus_queue.cc:237] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:30.704972 20129 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.705057 19712 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.705094 19445 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet ff26670b4bcc4167867fa54b97417365 with cas_config_opid_index 27: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:30.705248 20189 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.705317 19450 catalog_manager.cc:5649] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 27 to 28, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New cstate: current_term: 1 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.705329 20204 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.705677 20054 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.705935 20204 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.706272 19700 tablet_service.cc:1552] Processing DeleteTablet for tablet ff26670b4bcc4167867fa54b97417365 with delete_type TABLET_DATA_TOMBSTONED (TS ac203379ff4e4fdbac249bcd15cf5be1 not found in new config with opid_index 28) from {username='slave'} at 127.0.0.1:41418
I20251028 21:11:30.706591 20219 tablet_replica.cc:333] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:11:30.706740 20219 raft_consensus.cc:2243] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:30.706866 20219 raft_consensus.cc:2272] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:30.706995 19452 catalog_manager.cc:5649] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 28 to 29, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:30.707195 19712 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.706918 19443 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 4c67b4b90e724c8492a4c82e84487f23 with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:30.706933 20129 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.708441 19618 tablet_service.cc:1552] Processing DeleteTablet for tablet 4c67b4b90e724c8492a4c82e84487f23 with delete_type TABLET_DATA_TOMBSTONED (TS 6b99aaed3fec4c28bf3e3dbbd676ab65 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:30.708503 20219 ts_tablet_manager.cc:1916] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:30.708760 20220 tablet_replica.cc:333] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:11:30.708855 20220 raft_consensus.cc:2243] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:30.708932 20220 raft_consensus.cc:2272] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:30.709153 20220 ts_tablet_manager.cc:1916] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:30.709982 20219 ts_tablet_manager.cc:1929] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.27
I20251028 21:11:30.710041 20219 log.cc:1199] T ff26670b4bcc4167867fa54b97417365 P ac203379ff4e4fdbac249bcd15cf5be1: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/ff26670b4bcc4167867fa54b97417365
I20251028 21:11:30.710388 19442 catalog_manager.cc:4985] TS ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465): tablet ff26670b4bcc4167867fa54b97417365 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:30.710880 20220 ts_tablet_manager.cc:1929] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251028 21:11:30.710944 20220 log.cc:1199] T 4c67b4b90e724c8492a4c82e84487f23 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/4c67b4b90e724c8492a4c82e84487f23
I20251028 21:11:30.711318 19445 catalog_manager.cc:4985] TS 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457): tablet 4c67b4b90e724c8492a4c82e84487f23 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:30.749264 20189 raft_consensus.cc:1064] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: attempting to promote NON_VOTER a39fb1f187c544f2b77164040db04caf to VOTER
I20251028 21:11:30.749403 20189 consensus_queue.cc:237] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:30.749774 20129 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.749805 19631 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.749949 19712 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.749945 20205 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.750111 20204 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.750373 20204 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.750873 20204 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.750962 19631 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.751711 20129 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.751883 19449 catalog_manager.cc:5649] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.752496 19707 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.753476 19558 consensus_queue.cc:237] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:30.753793 20129 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.753803 19631 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.753968 20204 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.754099 20204 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.754492 20213 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.754686 19631 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.755302 19443 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet b22601bb9db44a3489691047e5a7c40a with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:30.755306 20129 raft_consensus.cc:2955] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:30.755389 19452 catalog_manager.cc:5649] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 28 to 29, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:30.755961 19700 tablet_service.cc:1552] Processing DeleteTablet for tablet b22601bb9db44a3489691047e5a7c40a with delete_type TABLET_DATA_TOMBSTONED (TS ac203379ff4e4fdbac249bcd15cf5be1 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:41418
I20251028 21:11:30.756210 20219 tablet_replica.cc:333] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:11:30.756278 20219 raft_consensus.cc:2243] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:30.756336 20219 raft_consensus.cc:2272] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:30.756529 20219 ts_tablet_manager.cc:1916] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:30.757652 20219 ts_tablet_manager.cc:1929] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251028 21:11:30.757705 20219 log.cc:1199] T b22601bb9db44a3489691047e5a7c40a P ac203379ff4e4fdbac249bcd15cf5be1: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/b22601bb9db44a3489691047e5a7c40a
I20251028 21:11:30.758069 19442 catalog_manager.cc:4985] TS ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465): tablet b22601bb9db44a3489691047e5a7c40a (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:30.773808 20226 raft_consensus.cc:1064] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: attempting to promote NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 to VOTER
I20251028 21:11:30.773991 20226 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:30.774402 20054 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 LEARNER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.774425 19631 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.774493 19558 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.774600 20228 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.774798 20225 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.774921 20226 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.775593 20228 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.775688 19559 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.776567 19631 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.776626 19450 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:30.775739 20054 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.777838 19707 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:30.778187 19631 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.778199 20054 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.778362 20225 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.778460 20188 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.778987 20226 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.779109 19631 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.779127 20054 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.779707 19442 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet e684839cc93d4e4685039496fbd01e03 with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:30.779924 19451 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:30.780483 19546 tablet_service.cc:1552] Processing DeleteTablet for tablet e684839cc93d4e4685039496fbd01e03 with delete_type TABLET_DATA_TOMBSTONED (TS 95350159593b4057ae155343f06246d0 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:53798
I20251028 21:11:30.780694 20232 tablet_replica.cc:333] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:11:30.780755 20232 raft_consensus.cc:2243] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:30.780819 20232 raft_consensus.cc:2272] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:30.780999 20232 ts_tablet_manager.cc:1916] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:30.782227 20232 ts_tablet_manager.cc:1929] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251028 21:11:30.782277 20232 log.cc:1199] T e684839cc93d4e4685039496fbd01e03 P 95350159593b4057ae155343f06246d0: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/e684839cc93d4e4685039496fbd01e03
I20251028 21:11:30.782604 19443 catalog_manager.cc:4985] TS 95350159593b4057ae155343f06246d0 (127.15.205.1:38429): tablet e684839cc93d4e4685039496fbd01e03 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:30.793805 20188 raft_consensus.cc:1064] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: attempting to promote NON_VOTER 9e69ab942ba042b6b83938c67405973c to VOTER
I20251028 21:11:30.793946 20188 consensus_queue.cc:237] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:30.794310 19978 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.794343 19559 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.794358 19631 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.794446 20225 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.794564 20225 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.794646 20225 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.795230 20188 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.795354 19978 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.795464 19631 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.796289 19559 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.796365 19450 catalog_manager.cc:5649] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c reported cstate change: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.796996 19707 consensus_queue.cc:237] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:30.797353 19978 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.797544 19631 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.797567 20188 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.797704 20188 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.797994 20226 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.798075 19978 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.798699 19442 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:30.798782 19450 catalog_manager.cc:5649] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:30.799319 19631 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.799408 19546 tablet_service.cc:1552] Processing DeleteTablet for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 with delete_type TABLET_DATA_TOMBSTONED (TS 95350159593b4057ae155343f06246d0 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:53798
I20251028 21:11:30.799479 20232 tablet_replica.cc:333] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:11:30.799530 20232 raft_consensus.cc:2243] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:30.799600 20232 raft_consensus.cc:2272] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:30.800323 20232 ts_tablet_manager.cc:1916] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:30.801463 20232 ts_tablet_manager.cc:1929] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251028 21:11:30.801517 20232 log.cc:1199] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 95350159593b4057ae155343f06246d0: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/f49b5a0ce6fe4d91aa0ab60a1d770949
I20251028 21:11:30.801784 19443 catalog_manager.cc:4985] TS 95350159593b4057ae155343f06246d0 (127.15.205.1:38429): tablet f49b5a0ce6fe4d91aa0ab60a1d770949 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:30.960884 20213 raft_consensus.cc:1064] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: attempting to promote NON_VOTER 9e69ab942ba042b6b83938c67405973c to VOTER
I20251028 21:11:30.961120 20213 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:30.961592 19631 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.961618 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.961694 19707 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:30.961834 20205 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.961946 20205 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.962023 20205 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:30.962561 20204 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.962662 19707 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.962879 19631 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.963574 19450 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.963995 19978 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.964444 19558 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:30.964870 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.964959 19707 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:30.965144 20189 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.965210 20241 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:30.965692 20213 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.965760 19707 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.965835 19978 raft_consensus.cc:2955] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:30.966554 19451 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 28 to 29, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New cstate: current_term: 1 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:30.966683 19443 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet e83bd2a0bd3c40c1b6abb84a48fa9013 with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:30.967080 19618 tablet_service.cc:1552] Processing DeleteTablet for tablet e83bd2a0bd3c40c1b6abb84a48fa9013 with delete_type TABLET_DATA_TOMBSTONED (TS 6b99aaed3fec4c28bf3e3dbbd676ab65 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:30.967175 20220 tablet_replica.cc:333] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:11:30.967234 20220 raft_consensus.cc:2243] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:30.967310 20220 raft_consensus.cc:2272] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:30.967581 20220 ts_tablet_manager.cc:1916] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:30.968528 20220 ts_tablet_manager.cc:1929] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251028 21:11:30.968568 20220 log.cc:1199] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/e83bd2a0bd3c40c1b6abb84a48fa9013
I20251028 21:11:30.968837 19445 catalog_manager.cc:4985] TS 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457): tablet e83bd2a0bd3c40c1b6abb84a48fa9013 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:31.573390 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:31.579236 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:31.579357 19707 raft_consensus.cc:606] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Received request to transfer leadership to TS 95350159593b4057ae155343f06246d0
I20251028 21:11:31.579671 19558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "4c67b4b90e724c8492a4c82e84487f23"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "a39fb1f187c544f2b77164040db04caf"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:31.579736 19558 raft_consensus.cc:606] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Received request to transfer leadership to TS a39fb1f187c544f2b77164040db04caf
I20251028 21:11:31.579981 19558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "8798d7cdf7864b0bbd9e5b2e26d0dc4f"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:31.580016 19558 raft_consensus.cc:606] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Received request to transfer leadership to TS 6b99aaed3fec4c28bf3e3dbbd676ab65
I20251028 21:11:31.582283 19631 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "903511324c64491b827a522851cb2402"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
new_leader_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:37102
I20251028 21:11:31.582340 19631 raft_consensus.cc:606] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Received request to transfer leadership to TS 95350159593b4057ae155343f06246d0
I20251028 21:11:31.582638 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "f49b5a0ce6fe4d91aa0ab60a1d770949"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "9e69ab942ba042b6b83938c67405973c"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:31.582685 19707 raft_consensus.cc:606] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Received request to transfer leadership to TS 9e69ab942ba042b6b83938c67405973c
I20251028 21:11:31.582937 19631 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "ff26670b4bcc4167867fa54b97417365"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
new_leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0"
from {username='slave'} at 127.0.0.1:37102
I20251028 21:11:31.582988 19631 raft_consensus.cc:606] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Received request to transfer leadership to TS 05d39c90d87d4cca9a23f40ad31b12e0
I20251028 21:11:31.583098 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20251028 21:11:31.583168 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:31.760234 20188 raft_consensus.cc:993] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:31.760329 20188 raft_consensus.cc:1081] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Signalling peer 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:31.760538 19558 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:53816
I20251028 21:11:31.760618 19558 raft_consensus.cc:493] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:31.760641 19558 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.761271 19558 raft_consensus.cc:515] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.761426 19558 leader_election.cc:290] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 2 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:31.761524 19631 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:31.761597 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:31.761602 19631 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.761662 19707 raft_consensus.cc:3055] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:31.761691 19707 raft_consensus.cc:740] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:31.761744 19707 consensus_queue.cc:260] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.761808 19707 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.762303 19631 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 2.
I20251028 21:11:31.762424 19707 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 2.
I20251028 21:11:31.762467 19527 leader_election.cc:304] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:31.762557 20204 raft_consensus.cc:2804] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:31.762634 20204 raft_consensus.cc:697] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:31.762686 20204 consensus_queue.cc:237] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.763139 19449 catalog_manager.cc:5649] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 1 to 2, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 2 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:31.803815 20203 raft_consensus.cc:993] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: : Instructing follower 05d39c90d87d4cca9a23f40ad31b12e0 to start an election
I20251028 21:11:31.803867 20203 raft_consensus.cc:1081] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Signalling peer 05d39c90d87d4cca9a23f40ad31b12e0 to start an election
I20251028 21:11:31.803990 20054 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "ff26670b4bcc4167867fa54b97417365"
dest_uuid: "05d39c90d87d4cca9a23f40ad31b12e0"
from {username='slave'} at 127.0.0.1:42766
I20251028 21:11:31.804037 20054 raft_consensus.cc:493] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:31.804054 20054 raft_consensus.cc:3060] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.804507 20054 raft_consensus.cc:515] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:31.804628 20054 leader_election.cc:290] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [CANDIDATE]: Term 2 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:31.804773 19631 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff26670b4bcc4167867fa54b97417365" candidate_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:31.804847 19631 raft_consensus.cc:3055] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:31.804872 19631 raft_consensus.cc:740] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:31.804926 19631 consensus_queue.cc:260] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:31.805006 19631 raft_consensus.cc:3060] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.805491 19631 raft_consensus.cc:2468] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d39c90d87d4cca9a23f40ad31b12e0 in term 2.
I20251028 21:11:31.805634 20020 leader_election.cc:304] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d39c90d87d4cca9a23f40ad31b12e0, 6b99aaed3fec4c28bf3e3dbbd676ab65; no voters:
I20251028 21:11:31.805754 20254 raft_consensus.cc:2804] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:31.805904 20254 raft_consensus.cc:697] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Becoming Leader. State: Replica: 05d39c90d87d4cca9a23f40ad31b12e0, State: Running, Role: LEADER
I20251028 21:11:31.805979 20254 consensus_queue.cc:237] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:31.806440 19449 catalog_manager.cc:5649] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: term changed from 1 to 2, leader changed from 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) to 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5). New cstate: current_term: 2 leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:31.807447 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff26670b4bcc4167867fa54b97417365" candidate_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 28 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:31.807500 19558 raft_consensus.cc:3060] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.807943 19558 raft_consensus.cc:2468] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d39c90d87d4cca9a23f40ad31b12e0 in term 2.
I20251028 21:11:31.831336 20246 raft_consensus.cc:993] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: : Instructing follower 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:31.831410 20246 raft_consensus.cc:1081] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Signalling peer 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:31.831569 19631 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "8798d7cdf7864b0bbd9e5b2e26d0dc4f"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:37070
I20251028 21:11:31.831615 19631 raft_consensus.cc:493] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:31.831631 19631 raft_consensus.cc:3060] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.832064 19631 raft_consensus.cc:515] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.832175 19631 leader_election.cc:290] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 2 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:31.832271 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8798d7cdf7864b0bbd9e5b2e26d0dc4f" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 2 candidate_status { last_received { term: 1 index: 25 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:31.832273 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8798d7cdf7864b0bbd9e5b2e26d0dc4f" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 2 candidate_status { last_received { term: 1 index: 25 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:31.832361 19707 raft_consensus.cc:3060] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.832391 19558 raft_consensus.cc:3055] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:31.832413 19558 raft_consensus.cc:740] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:31.832464 19558 consensus_queue.cc:260] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 25, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.832526 19558 raft_consensus.cc:3060] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.833024 19707 raft_consensus.cc:2468] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 2.
I20251028 21:11:31.833024 19558 raft_consensus.cc:2468] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 2.
I20251028 21:11:31.833201 19600 leader_election.cc:304] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:31.833287 20203 raft_consensus.cc:2804] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:31.833329 20203 raft_consensus.cc:697] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:31.833379 20203 consensus_queue.cc:237] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 25, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.833809 19449 catalog_manager.cc:5649] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 1 to 2, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:31.834084 20188 raft_consensus.cc:993] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 9e69ab942ba042b6b83938c67405973c to start an election
I20251028 21:11:31.834134 20188 raft_consensus.cc:1081] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Signalling peer 9e69ab942ba042b6b83938c67405973c to start an election
I20251028 21:11:31.834347 19978 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "f49b5a0ce6fe4d91aa0ab60a1d770949"
dest_uuid: "9e69ab942ba042b6b83938c67405973c"
from {username='slave'} at 127.0.0.1:60424
I20251028 21:11:31.834396 19978 raft_consensus.cc:493] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:31.834417 19978 raft_consensus.cc:3060] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.834805 19978 raft_consensus.cc:515] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:31.834924 19978 leader_election.cc:290] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [CANDIDATE]: Term 2 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:31.835223 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f49b5a0ce6fe4d91aa0ab60a1d770949" candidate_uuid: "9e69ab942ba042b6b83938c67405973c" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:31.835315 19707 raft_consensus.cc:3055] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:31.835350 19707 raft_consensus.cc:740] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:31.835399 19707 consensus_queue.cc:260] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:31.835462 19707 raft_consensus.cc:3060] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.835863 19707 raft_consensus.cc:2468] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e69ab942ba042b6b83938c67405973c in term 2.
I20251028 21:11:31.836033 19941 leader_election.cc:304] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e69ab942ba042b6b83938c67405973c, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:31.836205 20259 raft_consensus.cc:2804] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:31.836333 20259 raft_consensus.cc:697] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 2 LEADER]: Becoming Leader. State: Replica: 9e69ab942ba042b6b83938c67405973c, State: Running, Role: LEADER
I20251028 21:11:31.836392 20259 consensus_queue.cc:237] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:31.836767 19449 catalog_manager.cc:5649] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c reported cstate change: term changed from 1 to 2, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 9e69ab942ba042b6b83938c67405973c (127.15.205.4). New cstate: current_term: 2 leader_uuid: "9e69ab942ba042b6b83938c67405973c" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:31.837425 19631 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f49b5a0ce6fe4d91aa0ab60a1d770949" candidate_uuid: "9e69ab942ba042b6b83938c67405973c" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:31.837474 19631 raft_consensus.cc:3060] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.837872 19631 raft_consensus.cc:2468] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e69ab942ba042b6b83938c67405973c in term 2.
I20251028 21:11:31.905937 20246 raft_consensus.cc:993] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: : Instructing follower a39fb1f187c544f2b77164040db04caf to start an election
I20251028 21:11:31.905995 20246 raft_consensus.cc:1081] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Signalling peer a39fb1f187c544f2b77164040db04caf to start an election
I20251028 21:11:31.906162 20129 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "4c67b4b90e724c8492a4c82e84487f23"
dest_uuid: "a39fb1f187c544f2b77164040db04caf"
from {username='slave'} at 127.0.0.1:44218
I20251028 21:11:31.906214 20129 raft_consensus.cc:493] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:31.906234 20129 raft_consensus.cc:3060] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.906767 20129 raft_consensus.cc:515] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:31.906916 20129 leader_election.cc:290] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [CANDIDATE]: Term 2 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:31.907080 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c67b4b90e724c8492a4c82e84487f23" candidate_uuid: "a39fb1f187c544f2b77164040db04caf" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:31.907162 19558 raft_consensus.cc:3055] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:31.907188 19558 raft_consensus.cc:740] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:31.907233 19558 consensus_queue.cc:260] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:31.907366 19558 raft_consensus.cc:3060] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.907898 19558 raft_consensus.cc:2468] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a39fb1f187c544f2b77164040db04caf in term 2.
I20251028 21:11:31.908074 20093 leader_election.cc:304] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, a39fb1f187c544f2b77164040db04caf; no voters:
I20251028 21:11:31.908264 20263 raft_consensus.cc:2804] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:31.908380 20263 raft_consensus.cc:697] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 2 LEADER]: Becoming Leader. State: Replica: a39fb1f187c544f2b77164040db04caf, State: Running, Role: LEADER
I20251028 21:11:31.908450 20263 consensus_queue.cc:237] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:31.908905 19449 catalog_manager.cc:5649] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf reported cstate change: term changed from 1 to 2, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to a39fb1f187c544f2b77164040db04caf (127.15.205.6). New cstate: current_term: 2 leader_uuid: "a39fb1f187c544f2b77164040db04caf" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:31.909794 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c67b4b90e724c8492a4c82e84487f23" candidate_uuid: "a39fb1f187c544f2b77164040db04caf" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:31.909847 19707 raft_consensus.cc:3060] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:31.910269 19707 raft_consensus.cc:2468] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a39fb1f187c544f2b77164040db04caf in term 2.
I20251028 21:11:31.925792 20203 raft_consensus.cc:993] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: : Instructing follower 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:31.925889 20203 raft_consensus.cc:1081] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Signalling peer 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:31.926077 19558 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "903511324c64491b827a522851cb2402"
dest_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:53800
I20251028 21:11:31.926121 19558 raft_consensus.cc:493] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:31.926138 19558 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:31.926532 19558 raft_consensus.cc:515] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.926632 19558 leader_election.cc:290] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 3 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:31.926764 19631 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 27 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:31.926761 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 27 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:31.926847 19631 raft_consensus.cc:3055] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:11:31.926855 19707 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:31.926867 19631 raft_consensus.cc:740] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:31.926927 19631 consensus_queue.cc:260] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.926999 19631 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:31.927387 19707 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 3.
I20251028 21:11:31.927492 19631 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 3.
I20251028 21:11:31.927577 19524 leader_election.cc:304] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:31.927670 20246 raft_consensus.cc:2804] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:11:31.927724 20246 raft_consensus.cc:697] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:31.927773 20246 consensus_queue.cc:237] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:31.928231 19449 catalog_manager.cc:5649] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 2 to 3, leader changed from 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 3 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:31.967483 19707 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:31.967962 19631 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251028 21:11:31.967988 20054 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251028 21:11:31.968190 20245 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:31.968379 20268 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:31.968950 20188 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:31.969079 19631 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:31.970072 20054 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:31.970105 19449 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index 29 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:31.970845 19707 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:31.971333 19558 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
W20251028 21:11:31.971410 19675 consensus_peers.cc:597] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 -> Peer 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831): Couldn't send request to peer 9e69ab942ba042b6b83938c67405973c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d738dd2ed0444dfeb4ff5cafec9c1d2c. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:31.971560 20245 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
W20251028 21:11:31.971583 19675 consensus_peers.cc:597] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 -> Peer a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471): Couldn't send request to peer a39fb1f187c544f2b77164040db04caf. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e684839cc93d4e4685039496fbd01e03. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:31.971591 19631 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 1 index: 27. (index mismatch)
I20251028 21:11:31.971817 20245 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:31.972237 20188 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:31.972374 19558 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:31.972535 19631 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.27: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:31.973229 19450 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index -1 to 27, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251028 21:11:31.975890 19507 auto_rebalancer.cc:253] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251028 21:11:31.976766 19507 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet f49b5a0ce6fe4d91aa0ab60a1d770949, TS ac203379ff4e4fdbac249bcd15cf5be1 -> TS 05d39c90d87d4cca9a23f40ad31b12e0 move failed, destination replica disappeared from tablet's Raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
W20251028 21:11:31.976807 19507 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet f49b5a0ce6fe4d91aa0ab60a1d770949, TS ac203379ff4e4fdbac249bcd15cf5be1 -> TS 05d39c90d87d4cca9a23f40ad31b12e0 move failed, destination replica disappeared from tablet's Raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
W20251028 21:11:31.977422 19507 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 352c91b410644a06914dd52b1ba5c584, TS 6b99aaed3fec4c28bf3e3dbbd676ab65 -> TS 9e69ab942ba042b6b83938c67405973c move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
W20251028 21:11:31.977456 19507 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 352c91b410644a06914dd52b1ba5c584, TS 6b99aaed3fec4c28bf3e3dbbd676ab65 -> TS 9e69ab942ba042b6b83938c67405973c move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
W20251028 21:11:31.980110 19507 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet ff26670b4bcc4167867fa54b97417365, TS 95350159593b4057ae155343f06246d0 -> TS a39fb1f187c544f2b77164040db04caf move failed, destination replica disappeared from tablet's Raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
W20251028 21:11:31.980149 19507 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet ff26670b4bcc4167867fa54b97417365, TS 95350159593b4057ae155343f06246d0 -> TS a39fb1f187c544f2b77164040db04caf move failed, destination replica disappeared from tablet's Raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
W20251028 21:11:31.980911 19507 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 10b6448e96bb4fd08006249ba36d650e, TS ac203379ff4e4fdbac249bcd15cf5be1 -> TS 05d39c90d87d4cca9a23f40ad31b12e0 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
W20251028 21:11:31.980947 19507 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 10b6448e96bb4fd08006249ba36d650e, TS ac203379ff4e4fdbac249bcd15cf5be1 -> TS 05d39c90d87d4cca9a23f40ad31b12e0 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:32.261008 19558 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251028 21:11:32.261255 20254 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:32.262359 19631 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251028 21:11:32.262531 20281 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:32.302645 19558 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:32.302816 20263 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:32.303917 19707 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:32.304082 20263 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:32.328522 19631 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251028 21:11:32.328676 20246 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:32.329442 19707 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251028 21:11:32.329633 20246 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:32.357235 19631 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:32.357404 20291 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:32.358318 19707 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:32.358496 20293 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:32.369706 19712 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251028 21:11:32.369864 20204 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:32.370654 19631 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251028 21:11:32.370849 20204 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:32.427922 20299 ts_tablet_manager.cc:933] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: Initiating tablet copy from peer ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:32.428049 20299 tablet_copy_client.cc:323] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:42465
I20251028 21:11:32.428220 19722 tablet_copy_service.cc:140] P ac203379ff4e4fdbac249bcd15cf5be1: Received BeginTabletCopySession request for tablet e684839cc93d4e4685039496fbd01e03 from peer a39fb1f187c544f2b77164040db04caf ({username='slave'} at 127.0.0.1:41484)
I20251028 21:11:32.428268 19722 tablet_copy_service.cc:161] P ac203379ff4e4fdbac249bcd15cf5be1: Beginning new tablet copy session on tablet e684839cc93d4e4685039496fbd01e03 from peer a39fb1f187c544f2b77164040db04caf at {username='slave'} at 127.0.0.1:41484: session id = a39fb1f187c544f2b77164040db04caf-e684839cc93d4e4685039496fbd01e03
I20251028 21:11:32.428751 19722 tablet_copy_source_session.cc:215] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:32.429075 20299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e684839cc93d4e4685039496fbd01e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:32.430160 20299 tablet_copy_client.cc:806] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:32.430286 20299 tablet_copy_client.cc:670] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:32.430752 20300 tablet_copy_client.cc:1025] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:32.433815 19558 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 2 index: 26. (index mismatch)
I20251028 21:11:32.433952 20203 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251028 21:11:32.435019 19712 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 2 index: 26. (index mismatch)
I20251028 21:11:32.435276 20302 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251028 21:11:32.587129 20304 ts_tablet_manager.cc:933] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: Initiating tablet copy from peer ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:32.587342 20304 tablet_copy_client.cc:323] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:42465
I20251028 21:11:32.587491 19722 tablet_copy_service.cc:140] P ac203379ff4e4fdbac249bcd15cf5be1: Received BeginTabletCopySession request for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c from peer 9e69ab942ba042b6b83938c67405973c ({username='slave'} at 127.0.0.1:41448)
I20251028 21:11:32.587532 19722 tablet_copy_service.cc:161] P ac203379ff4e4fdbac249bcd15cf5be1: Beginning new tablet copy session on tablet d738dd2ed0444dfeb4ff5cafec9c1d2c from peer 9e69ab942ba042b6b83938c67405973c at {username='slave'} at 127.0.0.1:41448: session id = 9e69ab942ba042b6b83938c67405973c-d738dd2ed0444dfeb4ff5cafec9c1d2c
I20251028 21:11:32.588080 19722 tablet_copy_source_session.cc:215] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:32.588373 20304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:32.589758 20304 tablet_copy_client.cc:806] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:32.589887 20304 tablet_copy_client.cc:670] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:32.590370 20305 tablet_copy_client.cc:1025] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:33.583315 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:33.583840 19631 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:37102
I20251028 21:11:33.583950 19631 raft_consensus.cc:606] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:33.584249 19558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "b22601bb9db44a3489691047e5a7c40a"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:33.584321 19558 raft_consensus.cc:606] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Received request to transfer leadership to TS 6b99aaed3fec4c28bf3e3dbbd676ab65
I20251028 21:11:33.584576 19558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:33.584635 19558 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:33.584787 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251028 21:11:33.584870 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:33.709820 20307 raft_consensus.cc:993] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:33.709909 20307 raft_consensus.cc:1081] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:33.710058 19707 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41426
I20251028 21:11:33.710143 19707 raft_consensus.cc:493] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:33.710175 19707 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.710996 19707 raft_consensus.cc:515] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:33.711128 19707 leader_election.cc:290] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 2 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:33.711231 19631 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 2 candidate_status { last_received { term: 1 index: 25 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:33.711318 19631 raft_consensus.cc:3055] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:33.711333 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 2 candidate_status { last_received { term: 1 index: 25 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:33.711347 19631 raft_consensus.cc:740] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:33.711421 19558 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.711457 19631 consensus_queue.cc:260] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 25, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:33.711534 19631 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.711957 19558 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 2.
I20251028 21:11:33.712075 19631 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 2.
I20251028 21:11:33.712117 19676 leader_election.cc:304] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:33.712194 20245 raft_consensus.cc:2804] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:33.712239 20245 raft_consensus.cc:697] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:33.712291 20245 consensus_queue.cc:237] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 1.25, Last appended by leader: 25, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:33.712721 19451 catalog_manager.cc:5649] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 1 to 2, leader changed from 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 2 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } }
I20251028 21:11:33.771425 20246 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:33.771492 20246 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:33.771703 19712 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41428
I20251028 21:11:33.771763 19712 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:33.771790 19712 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.772526 19712 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:33.772652 19712 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 2 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:33.772758 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:33.772814 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.773217 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 2.
I20251028 21:11:33.773363 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:33.773365 19675 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e69ab942ba042b6b83938c67405973c, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:33.773432 19558 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:33.773454 19558 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:33.773499 20245 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:33.773561 20245 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:33.773509 19558 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:33.773614 20245 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:33.773643 19558 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.774259 19558 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 2.
I20251028 21:11:33.774443 19451 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 1 to 2, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 2 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:33.779019 20319 raft_consensus.cc:993] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: : Instructing follower 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:33.779086 20319 raft_consensus.cc:1081] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Signalling peer 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:33.779247 19631 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "b22601bb9db44a3489691047e5a7c40a"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:37070
I20251028 21:11:33.779325 19631 raft_consensus.cc:493] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:33.779372 19631 raft_consensus.cc:3060] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.779883 19631 raft_consensus.cc:515] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:33.779990 19631 leader_election.cc:290] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 2 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471)
I20251028 21:11:33.780141 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b22601bb9db44a3489691047e5a7c40a" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:33.780197 19558 raft_consensus.cc:3055] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:33.780220 19558 raft_consensus.cc:740] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:33.780269 19558 consensus_queue.cc:260] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:33.780344 19558 raft_consensus.cc:3060] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.780831 19558 raft_consensus.cc:2468] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 2.
I20251028 21:11:33.781008 19601 leader_election.cc:304] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:33.781088 20313 raft_consensus.cc:2804] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:33.781149 20313 raft_consensus.cc:697] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:33.781219 20313 consensus_queue.cc:237] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:33.781718 19451 catalog_manager.cc:5649] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 1 to 2, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:33.783784 20129 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b22601bb9db44a3489691047e5a7c40a" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "a39fb1f187c544f2b77164040db04caf"
I20251028 21:11:33.783844 20129 raft_consensus.cc:3060] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:33.784391 20129 raft_consensus.cc:2468] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 2.
I20251028 21:11:34.132635 19558 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 2 index: 26. (index mismatch)
I20251028 21:11:34.132833 20311 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251028 21:11:34.133905 19631 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 25. Preceding OpId from leader: term: 2 index: 26. (index mismatch)
I20251028 21:11:34.134104 20311 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251028 21:11:34.243002 20129 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:34.243213 20307 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:34.244233 19558 raft_consensus.cc:1275] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:34.244488 20307 consensus_queue.cc:1048] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:34.275971 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:34.276129 20311 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:34.276988 19558 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:34.277269 20311 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:34.430967 20299 tablet_copy_client.cc:538] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:34.432067 20299 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: Bootstrap starting.
I20251028 21:11:34.436580 20299 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=93 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:34.436784 20299 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: Bootstrap complete.
I20251028 21:11:34.436913 20299 ts_tablet_manager.cc:1403] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251028 21:11:34.437062 20299 raft_consensus.cc:359] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:34.437175 20299 raft_consensus.cc:740] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a39fb1f187c544f2b77164040db04caf, State: Initialized, Role: LEARNER
I20251028 21:11:34.437218 20299 consensus_queue.cc:260] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:34.437335 20299 ts_tablet_manager.cc:1434] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: Time spent starting tablet: real 0.000s user 0.000s sys 0.002s
I20251028 21:11:34.437548 19722 tablet_copy_service.cc:342] P ac203379ff4e4fdbac249bcd15cf5be1: Request end of tablet copy session a39fb1f187c544f2b77164040db04caf-e684839cc93d4e4685039496fbd01e03 received from {username='slave'} at 127.0.0.1:41484
I20251028 21:11:34.437609 19722 tablet_copy_service.cc:434] P ac203379ff4e4fdbac249bcd15cf5be1: ending tablet copy session a39fb1f187c544f2b77164040db04caf-e684839cc93d4e4685039496fbd01e03 on tablet e684839cc93d4e4685039496fbd01e03 with peer a39fb1f187c544f2b77164040db04caf
I20251028 21:11:34.590606 20304 tablet_copy_client.cc:538] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:34.591668 20304 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: Bootstrap starting.
I20251028 21:11:34.594534 20304 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=97 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:34.594674 20304 tablet_bootstrap.cc:492] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: Bootstrap complete.
I20251028 21:11:34.594748 20304 ts_tablet_manager.cc:1403] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 21:11:34.594841 20304 raft_consensus.cc:359] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:34.594897 20304 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9e69ab942ba042b6b83938c67405973c, State: Initialized, Role: LEARNER
I20251028 21:11:34.594916 20304 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:34.594973 20304 ts_tablet_manager.cc:1434] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:34.595139 19722 tablet_copy_service.cc:342] P ac203379ff4e4fdbac249bcd15cf5be1: Request end of tablet copy session 9e69ab942ba042b6b83938c67405973c-d738dd2ed0444dfeb4ff5cafec9c1d2c received from {username='slave'} at 127.0.0.1:41448
I20251028 21:11:34.595201 19722 tablet_copy_service.cc:434] P ac203379ff4e4fdbac249bcd15cf5be1: ending tablet copy session 9e69ab942ba042b6b83938c67405973c-d738dd2ed0444dfeb4ff5cafec9c1d2c on tablet d738dd2ed0444dfeb4ff5cafec9c1d2c with peer 9e69ab942ba042b6b83938c67405973c
I20251028 21:11:34.624094 19978 raft_consensus.cc:1217] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.26->[1.27-1.27] Dedup: 1.27->[]
I20251028 21:11:34.871460 20129 raft_consensus.cc:1217] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.29->[1.30-1.30] Dedup: 1.30->[]
I20251028 21:11:35.002275 20337 raft_consensus.cc:1064] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: attempting to promote NON_VOTER 9e69ab942ba042b6b83938c67405973c to VOTER
I20251028 21:11:35.002475 20337 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:35.002914 19978 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 1 LEARNER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:35.002918 19558 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:35.002914 19631 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20251028 21:11:35.003166 20245 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:35.003247 20340 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:35.003463 20340 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:35.004281 20245 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:35.004400 19978 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:35.005199 19631 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:35.005254 19558 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:35.005975 19451 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 27 to 28, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:35.007287 19707 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:35.007565 19978 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:35.007694 19631 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251028 21:11:35.007745 20337 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:35.007848 20340 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:35.008322 20337 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:35.008471 19978 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:35.008545 19631 raft_consensus.cc:2955] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:35.009130 19442 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:35.009275 19452 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c reported cstate change: config changed from index 28 to 29, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:35.009902 19546 tablet_service.cc:1552] Processing DeleteTablet for tablet d738dd2ed0444dfeb4ff5cafec9c1d2c with delete_type TABLET_DATA_TOMBSTONED (TS 95350159593b4057ae155343f06246d0 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:53798
I20251028 21:11:35.010077 20348 tablet_replica.cc:333] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:11:35.010166 20348 raft_consensus.cc:2243] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:35.010239 20348 raft_consensus.cc:2272] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:35.010470 20348 ts_tablet_manager.cc:1916] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:35.011865 20348 ts_tablet_manager.cc:1929] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.28
I20251028 21:11:35.011920 20348 log.cc:1199] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 95350159593b4057ae155343f06246d0: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/d738dd2ed0444dfeb4ff5cafec9c1d2c
I20251028 21:11:35.012221 19443 catalog_manager.cc:4985] TS 95350159593b4057ae155343f06246d0 (127.15.205.1:38429): tablet d738dd2ed0444dfeb4ff5cafec9c1d2c (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:35.446362 20245 raft_consensus.cc:1064] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: attempting to promote NON_VOTER a39fb1f187c544f2b77164040db04caf to VOTER
I20251028 21:11:35.446552 20245 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:35.446959 20054 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251028 21:11:35.446998 19631 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251028 21:11:35.447121 20129 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 1 LEARNER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251028 21:11:35.447257 20311 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:35.447362 20245 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:35.447544 20337 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:35.448128 20311 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:35.448223 20129 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:35.449062 19450 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf reported cstate change: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:35.449091 20054 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:35.449126 19631 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:35.449954 19712 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:35.450285 20054 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251028 21:11:35.450337 20129 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251028 21:11:35.450482 20245 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:35.450608 20337 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:35.450984 20337 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:35.451238 20054 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:35.451818 19442 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet e684839cc93d4e4685039496fbd01e03 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:35.451890 20129 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:35.452023 19452 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 31 to 32, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New cstate: current_term: 1 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:35.452596 19618 tablet_service.cc:1552] Processing DeleteTablet for tablet e684839cc93d4e4685039496fbd01e03 with delete_type TABLET_DATA_TOMBSTONED (TS 6b99aaed3fec4c28bf3e3dbbd676ab65 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:35.452809 20353 tablet_replica.cc:333] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:11:35.452956 20353 raft_consensus.cc:2243] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:35.453038 20353 raft_consensus.cc:2272] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:35.453258 20353 ts_tablet_manager.cc:1916] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:35.454178 20353 ts_tablet_manager.cc:1929] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.31
I20251028 21:11:35.454226 20353 log.cc:1199] T e684839cc93d4e4685039496fbd01e03 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/e684839cc93d4e4685039496fbd01e03
I20251028 21:11:35.454509 19445 catalog_manager.cc:4985] TS 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457): tablet e684839cc93d4e4685039496fbd01e03 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:35.585019 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:35.585389 19631 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "10b6448e96bb4fd08006249ba36d650e"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
new_leader_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:37102
I20251028 21:11:35.585457 19631 raft_consensus.cc:606] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Received request to transfer leadership to TS 95350159593b4057ae155343f06246d0
I20251028 21:11:35.585753 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:35.585836 19707 raft_consensus.cc:606] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Received request to transfer leadership to TS 6b99aaed3fec4c28bf3e3dbbd676ab65
I20251028 21:11:35.586164 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:35.586211 19707 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Received request to transfer leadership to TS 95350159593b4057ae155343f06246d0
I20251028 21:11:35.586310 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251028 21:11:35.586376 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:35.602883 20337 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:35.602939 20337 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Signalling peer 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:35.603091 19558 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:53816
I20251028 21:11:35.603143 19558 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:35.603161 19558 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:35.603704 19558 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:35.603806 19558 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 3 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:35.603955 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:35.604023 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:35.603953 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:35.604053 19707 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:11:35.604087 19707 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:35.604148 19707 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:35.604220 19707 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:35.604557 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 3.
I20251028 21:11:35.604744 19524 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, 9e69ab942ba042b6b83938c67405973c; no voters:
I20251028 21:11:35.604771 19707 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 3.
I20251028 21:11:35.604858 20319 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:11:35.604976 20319 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:35.605036 20319 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:35.605523 19450 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 2 to 3, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 3 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:35.733880 20337 raft_consensus.cc:993] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:35.733949 20337 raft_consensus.cc:1081] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Signalling peer 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:35.734091 19631 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:37080
I20251028 21:11:35.734171 19631 raft_consensus.cc:493] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:35.734191 19631 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:35.734725 19631 raft_consensus.cc:515] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:35.734844 19631 leader_election.cc:290] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 3 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:35.734997 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 3 candidate_status { last_received { term: 2 index: 26 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:35.734983 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 3 candidate_status { last_received { term: 2 index: 26 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:35.735080 19707 raft_consensus.cc:3055] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:11:35.735090 19558 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:35.735117 19707 raft_consensus.cc:740] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:35.735172 19707 consensus_queue.cc:260] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 2.26, Last appended by leader: 26, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:35.735256 19707 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:35.735649 19558 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 3.
I20251028 21:11:35.735775 19707 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 3.
I20251028 21:11:35.735823 19601 leader_election.cc:304] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:35.735909 20338 raft_consensus.cc:2804] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:11:35.736052 20338 raft_consensus.cc:697] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:35.736111 20338 consensus_queue.cc:237] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 2.26, Last appended by leader: 26, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:35.736618 19450 catalog_manager.cc:5649] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 2 to 3, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 3 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:35.879182 20338 raft_consensus.cc:993] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65: : Instructing follower 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:35.879299 20338 raft_consensus.cc:1081] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Signalling peer 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:35.879505 19558 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "10b6448e96bb4fd08006249ba36d650e"
dest_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:53800
I20251028 21:11:35.879570 19558 raft_consensus.cc:493] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:35.879590 19558 raft_consensus.cc:3060] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:35.880125 19558 raft_consensus.cc:515] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:35.880250 19558 leader_election.cc:290] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 2 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:35.880386 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "10b6448e96bb4fd08006249ba36d650e" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:35.880386 19631 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "10b6448e96bb4fd08006249ba36d650e" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 26 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:35.880478 19707 raft_consensus.cc:3060] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:35.880486 19631 raft_consensus.cc:3055] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:35.880530 19631 raft_consensus.cc:740] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:35.880592 19631 consensus_queue.cc:260] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:35.880667 19631 raft_consensus.cc:3060] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:35.881166 19707 raft_consensus.cc:2468] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 2.
I20251028 21:11:35.881300 19631 raft_consensus.cc:2468] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 2.
I20251028 21:11:35.881326 19524 leader_election.cc:304] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:35.881460 20319 raft_consensus.cc:2804] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:35.881529 20319 raft_consensus.cc:697] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:35.881579 20319 consensus_queue.cc:237] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 1.26, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } }
I20251028 21:11:35.882093 19450 catalog_manager.cc:5649] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 1 to 2, leader changed from 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 2 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:36.010200 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251028 21:11:36.010411 20319 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:36.011513 19707 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251028 21:11:36.011695 20319 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:36.184162 19707 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 26. Preceding OpId from leader: term: 3 index: 27. (index mismatch)
I20251028 21:11:36.184402 20338 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:36.185529 19558 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 26. Preceding OpId from leader: term: 3 index: 27. (index mismatch)
I20251028 21:11:36.185693 20338 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:36.441767 19631 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251028 21:11:36.442008 20319 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:36.443073 19707 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251028 21:11:36.443233 20369 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:36.453639 19631 consensus_queue.cc:237] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 2.26, Last appended by leader: 25, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:36.454120 19558 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
I20251028 21:11:36.454128 19707 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 26. Preceding OpId from leader: term: 2 index: 27. (index mismatch)
W20251028 21:11:36.454192 19601 consensus_peers.cc:597] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 -> Peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477): Couldn't send request to peer 05d39c90d87d4cca9a23f40ad31b12e0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8798d7cdf7864b0bbd9e5b2e26d0dc4f. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:36.454269 20338 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:36.454350 20356 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20251028 21:11:36.455003 20356 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Committing config change with OpId 2.27: config changed from index -1 to 27, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.455108 19707 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.27: config changed from index -1 to 27, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.455145 19558 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.27: config changed from index -1 to 27, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.456099 19452 catalog_manager.cc:5649] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index -1 to 27, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.458757 20129 consensus_queue.cc:237] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:36.459278 19707 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:36.459419 19558 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:36.459578 20309 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:36.459666 20380 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:36.460321 20380 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.460412 19558 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.461483 19707 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.461524 19631 consensus_queue.cc:237] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 3.27, Last appended by leader: 26, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:36.461539 19450 catalog_manager.cc:5649] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 29 to 31, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New cstate: current_term: 2 leader_uuid: "a39fb1f187c544f2b77164040db04caf" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.462054 19712 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 3 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251028 21:11:36.462244 19558 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 3 index: 27. Preceding OpId from leader: term: 3 index: 28. (index mismatch)
I20251028 21:11:36.462257 20356 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20251028 21:11:36.462428 19600 consensus_peers.cc:597] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 -> Peer a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471): Couldn't send request to peer a39fb1f187c544f2b77164040db04caf. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 590f65c9d19a4688b25734528ac7474f. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:36.462512 20356 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:36.462853 20338 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Committing config change with OpId 3.28: config changed from index -1 to 28, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:36.463061 19558 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Committing config change with OpId 3.28: config changed from index -1 to 28, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
W20251028 21:11:36.463142 20093 consensus_peers.cc:597] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf -> Peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477): Couldn't send request to peer 05d39c90d87d4cca9a23f40ad31b12e0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c67b4b90e724c8492a4c82e84487f23. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:36.463428 19707 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Committing config change with OpId 3.28: config changed from index -1 to 28, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:36.463902 19449 catalog_manager.cc:5649] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index -1 to 28, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New cstate: current_term: 3 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:36.463989 19558 consensus_queue.cc:237] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 3.28, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:36.464414 19631 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251028 21:11:36.464432 19707 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251028 21:11:36.464568 20369 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20251028 21:11:36.464656 19524 consensus_peers.cc:597] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 -> Peer 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831): Couldn't send request to peer 9e69ab942ba042b6b83938c67405973c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 903511324c64491b827a522851cb2402. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:36.464735 20369 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:36.465278 20354 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:36.465389 19707 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:36.465983 19631 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Committing config change with OpId 3.29: config changed from index -1 to 29, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:36.466671 19452 catalog_manager.cc:5649] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index -1 to 29, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New cstate: current_term: 3 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:36.466756 19558 consensus_queue.cc:237] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:36.467232 19707 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251028 21:11:36.467422 20319 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:36.467240 19631 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251028 21:11:36.467614 20319 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:36.468226 20319 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.469183 19631 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.469358 19707 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:36.469877 19449 catalog_manager.cc:5649] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index -1 to 28, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New cstate: current_term: 2 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:36.470248 20054 consensus_queue.cc:237] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
W20251028 21:11:36.470523 19525 consensus_peers.cc:597] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 -> Peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477): Couldn't send request to peer 05d39c90d87d4cca9a23f40ad31b12e0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 352c91b410644a06914dd52b1ba5c584. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:36.470767 19558 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:36.470767 19631 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:36.471212 20254 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:36.471347 20254 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:36.471964 20394 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index 28 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:36.472113 19558 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 28 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:36.472826 19631 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 28 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:36.472854 19449 catalog_manager.cc:5649] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 28 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New cstate: current_term: 2 leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
W20251028 21:11:36.473659 20017 consensus_peers.cc:597] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 -> Peer a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471): Couldn't send request to peer a39fb1f187c544f2b77164040db04caf. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff26670b4bcc4167867fa54b97417365. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:36.832491 20399 ts_tablet_manager.cc:933] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: Initiating tablet copy from peer 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:36.832700 20399 tablet_copy_client.cc:323] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:36457
I20251028 21:11:36.832873 19647 tablet_copy_service.cc:140] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Received BeginTabletCopySession request for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f from peer 05d39c90d87d4cca9a23f40ad31b12e0 ({username='slave'} at 127.0.0.1:37100)
I20251028 21:11:36.832928 19647 tablet_copy_service.cc:161] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Beginning new tablet copy session on tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f from peer 05d39c90d87d4cca9a23f40ad31b12e0 at {username='slave'} at 127.0.0.1:37100: session id = 05d39c90d87d4cca9a23f40ad31b12e0-8798d7cdf7864b0bbd9e5b2e26d0dc4f
I20251028 21:11:36.833454 19647 tablet_copy_source_session.cc:215] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:36.833743 20399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:36.835084 20399 tablet_copy_client.cc:806] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:36.835300 20399 tablet_copy_client.cc:670] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:36.835728 20400 tablet_copy_client.cc:1025] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:36.888206 20401 ts_tablet_manager.cc:933] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: Initiating tablet copy from peer 95350159593b4057ae155343f06246d0 (127.15.205.1:38429)
I20251028 21:11:36.888412 20401 tablet_copy_client.cc:323] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.1:38429
I20251028 21:11:36.888573 19571 tablet_copy_service.cc:140] P 95350159593b4057ae155343f06246d0: Received BeginTabletCopySession request for tablet 352c91b410644a06914dd52b1ba5c584 from peer 05d39c90d87d4cca9a23f40ad31b12e0 ({username='slave'} at 127.0.0.1:53862)
I20251028 21:11:36.888636 19571 tablet_copy_service.cc:161] P 95350159593b4057ae155343f06246d0: Beginning new tablet copy session on tablet 352c91b410644a06914dd52b1ba5c584 from peer 05d39c90d87d4cca9a23f40ad31b12e0 at {username='slave'} at 127.0.0.1:53862: session id = 05d39c90d87d4cca9a23f40ad31b12e0-352c91b410644a06914dd52b1ba5c584
I20251028 21:11:36.889101 19571 tablet_copy_source_session.cc:215] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:36.889354 20401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 352c91b410644a06914dd52b1ba5c584. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:36.890551 20401 tablet_copy_client.cc:806] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:36.890679 20401 tablet_copy_client.cc:670] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:36.891232 20402 tablet_copy_client.cc:1025] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:36.891441 20403 ts_tablet_manager.cc:933] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: Initiating tablet copy from peer 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:36.891584 20403 tablet_copy_client.cc:323] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:36457
I20251028 21:11:36.893796 19647 tablet_copy_service.cc:140] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Received BeginTabletCopySession request for tablet 590f65c9d19a4688b25734528ac7474f from peer a39fb1f187c544f2b77164040db04caf ({username='slave'} at 127.0.0.1:53888)
I20251028 21:11:36.893843 19647 tablet_copy_service.cc:161] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Beginning new tablet copy session on tablet 590f65c9d19a4688b25734528ac7474f from peer a39fb1f187c544f2b77164040db04caf at {username='slave'} at 127.0.0.1:53888: session id = a39fb1f187c544f2b77164040db04caf-590f65c9d19a4688b25734528ac7474f
I20251028 21:11:36.894186 19647 tablet_copy_source_session.cc:215] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:36.894454 20403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590f65c9d19a4688b25734528ac7474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:36.895707 20403 tablet_copy_client.cc:806] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:36.895848 20403 tablet_copy_client.cc:670] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:36.896303 20404 tablet_copy_client.cc:1025] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:36.896804 20406 ts_tablet_manager.cc:933] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: Initiating tablet copy from peer 95350159593b4057ae155343f06246d0 (127.15.205.1:38429)
I20251028 21:11:36.896953 20406 tablet_copy_client.cc:323] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.1:38429
I20251028 21:11:36.897101 19571 tablet_copy_service.cc:140] P 95350159593b4057ae155343f06246d0: Received BeginTabletCopySession request for tablet 903511324c64491b827a522851cb2402 from peer 9e69ab942ba042b6b83938c67405973c ({username='slave'} at 127.0.0.1:53846)
I20251028 21:11:36.897149 19571 tablet_copy_service.cc:161] P 95350159593b4057ae155343f06246d0: Beginning new tablet copy session on tablet 903511324c64491b827a522851cb2402 from peer 9e69ab942ba042b6b83938c67405973c at {username='slave'} at 127.0.0.1:53846: session id = 9e69ab942ba042b6b83938c67405973c-903511324c64491b827a522851cb2402
I20251028 21:11:36.897596 19571 tablet_copy_source_session.cc:215] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:36.897811 20406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903511324c64491b827a522851cb2402. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:36.898852 20406 tablet_copy_client.cc:806] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:36.898983 20406 tablet_copy_client.cc:670] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:36.899457 20407 tablet_copy_client.cc:1025] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:37.015771 20409 ts_tablet_manager.cc:933] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: Initiating tablet copy from peer a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471)
I20251028 21:11:37.015956 20409 tablet_copy_client.cc:323] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.6:45471
I20251028 21:11:37.016136 20139 tablet_copy_service.cc:140] P a39fb1f187c544f2b77164040db04caf: Received BeginTabletCopySession request for tablet 4c67b4b90e724c8492a4c82e84487f23 from peer 05d39c90d87d4cca9a23f40ad31b12e0 ({username='slave'} at 127.0.0.1:47826)
I20251028 21:11:37.016196 20139 tablet_copy_service.cc:161] P a39fb1f187c544f2b77164040db04caf: Beginning new tablet copy session on tablet 4c67b4b90e724c8492a4c82e84487f23 from peer 05d39c90d87d4cca9a23f40ad31b12e0 at {username='slave'} at 127.0.0.1:47826: session id = 05d39c90d87d4cca9a23f40ad31b12e0-4c67b4b90e724c8492a4c82e84487f23
I20251028 21:11:37.016726 20139 tablet_copy_source_session.cc:215] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:37.017010 20409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c67b4b90e724c8492a4c82e84487f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:37.018316 20409 tablet_copy_client.cc:806] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:37.018431 20409 tablet_copy_client.cc:670] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:37.018918 20410 tablet_copy_client.cc:1025] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:37.079962 20411 ts_tablet_manager.cc:933] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: Initiating tablet copy from peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477)
I20251028 21:11:37.080118 20411 tablet_copy_client.cc:323] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.5:44477
I20251028 21:11:37.080312 20064 tablet_copy_service.cc:140] P 05d39c90d87d4cca9a23f40ad31b12e0: Received BeginTabletCopySession request for tablet ff26670b4bcc4167867fa54b97417365 from peer a39fb1f187c544f2b77164040db04caf ({username='slave'} at 127.0.0.1:47490)
I20251028 21:11:37.080371 20064 tablet_copy_service.cc:161] P 05d39c90d87d4cca9a23f40ad31b12e0: Beginning new tablet copy session on tablet ff26670b4bcc4167867fa54b97417365 from peer a39fb1f187c544f2b77164040db04caf at {username='slave'} at 127.0.0.1:47490: session id = a39fb1f187c544f2b77164040db04caf-ff26670b4bcc4167867fa54b97417365
I20251028 21:11:37.080948 20064 tablet_copy_source_session.cc:215] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:37.081233 20411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff26670b4bcc4167867fa54b97417365. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:37.082453 20411 tablet_copy_client.cc:806] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:37.082571 20411 tablet_copy_client.cc:670] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:37.083025 20412 tablet_copy_client.cc:1025] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:37.586520 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:37.586902 19558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:37.587003 19558 raft_consensus.cc:606] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Received request to transfer leadership to TS 6b99aaed3fec4c28bf3e3dbbd676ab65
I20251028 21:11:37.587321 19633 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:37102
I20251028 21:11:37.587397 19633 raft_consensus.cc:606] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:37.587685 19558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:37.587754 19558 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:37.587919 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251028 21:11:37.588008 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:37.830228 20408 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:37.830317 20408 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:37.830495 19707 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41428
I20251028 21:11:37.830569 19707 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:37.830605 19707 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:37.831455 19707 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:37.831593 19707 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 4 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:37.831897 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:37.831986 19558 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Stepping down as leader of term 3
I20251028 21:11:37.831969 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:37.832036 19558 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:37.832049 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:37.832091 19558 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:37.832165 19558 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:37.832607 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 4.
I20251028 21:11:37.832691 19558 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 4.
I20251028 21:11:37.832760 19675 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e69ab942ba042b6b83938c67405973c, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:37.832829 20311 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Leader election won for term 4
I20251028 21:11:37.832919 20311 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:37.832963 20311 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:37.833418 19450 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 3 to 4, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 4 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:37.851222 20356 raft_consensus.cc:993] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:37.851311 20415 raft_consensus.cc:1081] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:37.851464 19707 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41426
I20251028 21:11:37.851517 19707 raft_consensus.cc:493] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:37.851545 19707 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:37.852046 19707 raft_consensus.cc:515] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:37.852147 19707 leader_election.cc:290] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 4 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:37.852244 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 4 candidate_status { last_received { term: 3 index: 28 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:37.852308 19558 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:37.852294 19633 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "590f65c9d19a4688b25734528ac7474f" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 4 candidate_status { last_received { term: 3 index: 28 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:37.852365 19633 raft_consensus.cc:3055] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Stepping down as leader of term 3
I20251028 21:11:37.852388 19633 raft_consensus.cc:740] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:37.852427 19633 consensus_queue.cc:260] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 3.28, Last appended by leader: 28, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:37.852514 19633 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:37.852933 19558 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 4.
I20251028 21:11:37.853072 19676 leader_election.cc:304] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:37.853107 19633 raft_consensus.cc:2468] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 4.
I20251028 21:11:37.853142 20311 raft_consensus.cc:2804] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Leader election won for term 4
I20251028 21:11:37.853175 20311 raft_consensus.cc:697] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:37.853210 20311 consensus_queue.cc:237] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 3.28, Last appended by leader: 28, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:37.853601 19450 catalog_manager.cc:5649] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 3 to 4, leader changed from 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 4 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:37.979935 20354 raft_consensus.cc:993] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0: : Instructing follower 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:37.980016 20354 raft_consensus.cc:1081] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Signalling peer 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:37.980145 19633 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:37070
I20251028 21:11:37.980211 19633 raft_consensus.cc:493] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:37.980239 19633 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:37.980932 19633 raft_consensus.cc:515] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:37.981084 19633 leader_election.cc:290] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 3 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:37.981195 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:37.981259 19712 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 3 candidate_status { last_received { term: 2 index: 28 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:37.981271 19558 raft_consensus.cc:3055] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:11:37.981338 19712 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:37.981406 19558 raft_consensus.cc:740] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:37.981453 19558 consensus_queue.cc:260] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:37.981545 19558 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:37.982079 19712 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 3.
I20251028 21:11:37.982162 19558 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 3.
I20251028 21:11:37.982209 19600 leader_election.cc:304] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:37.982276 20415 raft_consensus.cc:2804] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:11:37.982327 20415 raft_consensus.cc:697] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:37.982367 20415 consensus_queue.cc:237] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:37.982831 19450 catalog_manager.cc:5649] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 2 to 3, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 3 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:37.984649 19633 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:37.984709 19633 raft_consensus.cc:606] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Received request to transfer leadership
I20251028 21:11:37.986159 19633 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:37.986217 19633 raft_consensus.cc:606] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Received request to transfer leadership
W20251028 21:11:37.986332 19445 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 352c91b410644a06914dd52b1ba5c584 already in progress [suppressed 3 similar messages]
I20251028 21:11:37.986647 19633 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:37.986701 19633 raft_consensus.cc:606] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Received request to transfer leadership
W20251028 21:11:37.986846 19507 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 352c91b410644a06914dd52b1ba5c584 already in progress
W20251028 21:11:37.986896 19507 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 352c91b410644a06914dd52b1ba5c584 already in progress
I20251028 21:11:38.229799 19558 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 4 index: 29. (index mismatch)
I20251028 21:11:38.229997 20311 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20251028 21:11:38.231515 19675 consensus_peers.cc:597] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 -> Peer a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471): Couldn't send request to peer a39fb1f187c544f2b77164040db04caf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:38.231549 19633 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 3 index: 28. Preceding OpId from leader: term: 4 index: 29. (index mismatch)
I20251028 21:11:38.231781 20311 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:38.385749 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251028 21:11:38.385968 20419 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:38.386722 19558 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251028 21:11:38.386916 20419 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
W20251028 21:11:38.400985 19601 consensus_peers.cc:597] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 -> Peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477): Couldn't send request to peer 05d39c90d87d4cca9a23f40ad31b12e0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:38.439460 19712 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251028 21:11:38.439666 20415 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:38.440402 20415 raft_consensus.cc:993] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:38.440448 20415 raft_consensus.cc:1081] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:38.440788 19712 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41426
I20251028 21:11:38.440816 19558 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251028 21:11:38.440886 19712 raft_consensus.cc:493] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:38.440920 19712 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:38.441159 20439 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:38.441784 19712 raft_consensus.cc:515] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:38.441928 19712 leader_election.cc:290] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 4 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:38.442054 19633 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:38.442113 19633 raft_consensus.cc:3055] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Stepping down as leader of term 3
I20251028 21:11:38.442140 19633 raft_consensus.cc:740] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:38.442180 19633 consensus_queue.cc:260] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:38.442250 19633 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:38.442721 19633 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 4.
I20251028 21:11:38.442874 19678 leader_election.cc:304] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:38.443001 20419 raft_consensus.cc:2804] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Leader election won for term 4
I20251028 21:11:38.443059 20419 raft_consensus.cc:697] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:38.443110 20419 consensus_queue.cc:237] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:38.443559 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 4 candidate_status { last_received { term: 3 index: 29 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:38.443619 19558 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:38.443630 19449 catalog_manager.cc:5649] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 3 to 4, leader changed from 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 4 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:38.444258 19558 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 4.
I20251028 21:11:38.824666 19558 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251028 21:11:38.824869 20311 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:38.825522 19633 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251028 21:11:38.825780 20419 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
W20251028 21:11:38.825996 19676 consensus_peers.cc:597] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 -> Peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477): Couldn't send request to peer 05d39c90d87d4cca9a23f40ad31b12e0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:38.835938 20399 tablet_copy_client.cc:538] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:38.836905 20399 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap starting.
I20251028 21:11:38.840523 20399 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=111 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:38.840703 20399 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap complete.
I20251028 21:11:38.840795 20399 ts_tablet_manager.cc:1403] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251028 21:11:38.840955 20399 raft_consensus.cc:359] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:38.841027 20399 raft_consensus.cc:740] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05d39c90d87d4cca9a23f40ad31b12e0, State: Initialized, Role: LEARNER
I20251028 21:11:38.841058 20399 consensus_queue.cc:260] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 2.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 27 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:38.841181 20399 ts_tablet_manager.cc:1434] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.002s
I20251028 21:11:38.841380 19647 tablet_copy_service.cc:342] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Request end of tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-8798d7cdf7864b0bbd9e5b2e26d0dc4f received from {username='slave'} at 127.0.0.1:37100
I20251028 21:11:38.841432 19647 tablet_copy_service.cc:434] P 6b99aaed3fec4c28bf3e3dbbd676ab65: ending tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-8798d7cdf7864b0bbd9e5b2e26d0dc4f on tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f with peer 05d39c90d87d4cca9a23f40ad31b12e0
I20251028 21:11:38.891438 20401 tablet_copy_client.cc:538] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:38.892254 20401 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap starting.
I20251028 21:11:38.895905 20401 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=92 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:38.896077 20401 tablet_bootstrap.cc:492] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap complete.
I20251028 21:11:38.896168 20401 ts_tablet_manager.cc:1403] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251028 21:11:38.896304 20401 raft_consensus.cc:359] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:38.896368 20401 raft_consensus.cc:740] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05d39c90d87d4cca9a23f40ad31b12e0, State: Initialized, Role: LEARNER
I20251028 21:11:38.896397 20401 consensus_queue.cc:260] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:38.896481 20403 tablet_copy_client.cc:538] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:38.896550 20401 ts_tablet_manager.cc:1434] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:11:38.897157 20403 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: Bootstrap starting.
I20251028 21:11:38.898586 19571 tablet_copy_service.cc:342] P 95350159593b4057ae155343f06246d0: Request end of tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-352c91b410644a06914dd52b1ba5c584 received from {username='slave'} at 127.0.0.1:53862
I20251028 21:11:38.898625 19571 tablet_copy_service.cc:434] P 95350159593b4057ae155343f06246d0: ending tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-352c91b410644a06914dd52b1ba5c584 on tablet 352c91b410644a06914dd52b1ba5c584 with peer 05d39c90d87d4cca9a23f40ad31b12e0
I20251028 21:11:38.901257 20403 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=116 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:38.901430 20403 tablet_bootstrap.cc:492] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: Bootstrap complete.
I20251028 21:11:38.901530 20403 ts_tablet_manager.cc:1403] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251028 21:11:38.901674 20403 raft_consensus.cc:359] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:38.901749 20403 raft_consensus.cc:740] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a39fb1f187c544f2b77164040db04caf, State: Initialized, Role: LEARNER
I20251028 21:11:38.901789 20403 consensus_queue.cc:260] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 3.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:38.901870 20403 ts_tablet_manager.cc:1434] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:38.901935 20406 tablet_copy_client.cc:538] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:38.902634 20406 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: Bootstrap starting.
I20251028 21:11:38.903470 19647 tablet_copy_service.cc:342] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Request end of tablet copy session a39fb1f187c544f2b77164040db04caf-590f65c9d19a4688b25734528ac7474f received from {username='slave'} at 127.0.0.1:53888
I20251028 21:11:38.903517 19647 tablet_copy_service.cc:434] P 6b99aaed3fec4c28bf3e3dbbd676ab65: ending tablet copy session a39fb1f187c544f2b77164040db04caf-590f65c9d19a4688b25734528ac7474f on tablet 590f65c9d19a4688b25734528ac7474f with peer a39fb1f187c544f2b77164040db04caf
I20251028 21:11:38.906646 20406 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:38.906858 20406 tablet_bootstrap.cc:492] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: Bootstrap complete.
I20251028 21:11:38.906957 20406 ts_tablet_manager.cc:1403] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251028 21:11:38.907140 20406 raft_consensus.cc:359] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:38.907208 20406 raft_consensus.cc:740] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 9e69ab942ba042b6b83938c67405973c, State: Initialized, Role: LEARNER
I20251028 21:11:38.907250 20406 consensus_queue.cc:260] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 3.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:38.907357 20406 ts_tablet_manager.cc:1434] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:38.907575 19571 tablet_copy_service.cc:342] P 95350159593b4057ae155343f06246d0: Request end of tablet copy session 9e69ab942ba042b6b83938c67405973c-903511324c64491b827a522851cb2402 received from {username='slave'} at 127.0.0.1:53846
I20251028 21:11:38.907619 19571 tablet_copy_service.cc:434] P 95350159593b4057ae155343f06246d0: ending tablet copy session 9e69ab942ba042b6b83938c67405973c-903511324c64491b827a522851cb2402 on tablet 903511324c64491b827a522851cb2402 with peer 9e69ab942ba042b6b83938c67405973c
I20251028 21:11:39.019104 20409 tablet_copy_client.cc:538] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:39.019984 20409 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap starting.
I20251028 21:11:39.024355 20409 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=110 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:39.024530 20409 tablet_bootstrap.cc:492] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap complete.
I20251028 21:11:39.024621 20409 ts_tablet_manager.cc:1403] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251028 21:11:39.024760 20409 raft_consensus.cc:359] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:39.024852 20409 raft_consensus.cc:740] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05d39c90d87d4cca9a23f40ad31b12e0, State: Initialized, Role: LEARNER
I20251028 21:11:39.024916 20409 consensus_queue.cc:260] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:39.025007 20409 ts_tablet_manager.cc:1434] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.002s
I20251028 21:11:39.025156 20139 tablet_copy_service.cc:342] P a39fb1f187c544f2b77164040db04caf: Request end of tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-4c67b4b90e724c8492a4c82e84487f23 received from {username='slave'} at 127.0.0.1:47826
I20251028 21:11:39.025220 20139 tablet_copy_service.cc:434] P a39fb1f187c544f2b77164040db04caf: ending tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-4c67b4b90e724c8492a4c82e84487f23 on tablet 4c67b4b90e724c8492a4c82e84487f23 with peer 05d39c90d87d4cca9a23f40ad31b12e0
I20251028 21:11:39.065580 20129 raft_consensus.cc:3060] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 3 LEARNER]: Advancing to term 4
I20251028 21:11:39.083202 20411 tablet_copy_client.cc:538] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:39.083803 20411 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: Bootstrap starting.
I20251028 21:11:39.087281 20411 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=102 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:39.087491 20411 tablet_bootstrap.cc:492] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: Bootstrap complete.
I20251028 21:11:39.087590 20411 ts_tablet_manager.cc:1403] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20251028 21:11:39.087877 20411 raft_consensus.cc:359] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:39.087970 20411 raft_consensus.cc:740] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a39fb1f187c544f2b77164040db04caf, State: Initialized, Role: LEARNER
I20251028 21:11:39.087999 20411 consensus_queue.cc:260] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:39.088092 20411 ts_tablet_manager.cc:1434] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:39.088260 20064 tablet_copy_service.cc:342] P 05d39c90d87d4cca9a23f40ad31b12e0: Request end of tablet copy session a39fb1f187c544f2b77164040db04caf-ff26670b4bcc4167867fa54b97417365 received from {username='slave'} at 127.0.0.1:47490
I20251028 21:11:39.088337 20064 tablet_copy_service.cc:434] P 05d39c90d87d4cca9a23f40ad31b12e0: ending tablet copy session a39fb1f187c544f2b77164040db04caf-ff26670b4bcc4167867fa54b97417365 on tablet ff26670b4bcc4167867fa54b97417365 with peer a39fb1f187c544f2b77164040db04caf
I20251028 21:11:39.122836 20053 raft_consensus.cc:1217] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.26->[2.27-2.27] Dedup: 2.27->[]
I20251028 21:11:39.178543 20129 raft_consensus.cc:1217] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 LEARNER]: Deduplicated request from leader. Original: 2.29->[2.30-2.30] Dedup: 2.30->[]
I20251028 21:11:39.409369 19978 raft_consensus.cc:1217] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 LEARNER]: Deduplicated request from leader. Original: 3.28->[3.29-3.29] Dedup: 3.29->[]
I20251028 21:11:39.423632 20053 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Advancing to term 4
I20251028 21:11:39.424437 20053 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 4 LEARNER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 3 index: 29. (index mismatch)
I20251028 21:11:39.424594 20311 consensus_queue.cc:1050] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:39.425329 20311 raft_consensus.cc:1064] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: attempting to promote NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 to VOTER
I20251028 21:11:39.425487 20311 consensus_queue.cc:237] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 4.30, Last appended by leader: 29, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:39.425928 19631 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251028 21:11:39.425928 19559 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251028 21:11:39.426052 20054 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 4 LEARNER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251028 21:11:39.426128 20419 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.426270 20453 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.426390 20419 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.426910 20419 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Committing config change with OpId 4.31: config changed from index 28 to 31, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.427047 20054 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.427846 19559 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.427982 19631 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 28 to 31, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.428038 19450 catalog_manager.cc:5649] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: config changed from index 28 to 31, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.429198 19707 consensus_queue.cc:237] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 4.31, Last appended by leader: 29, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:39.429507 20054 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251028 21:11:39.429600 19559 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251028 21:11:39.429715 20441 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:39.429795 20453 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:39.430270 20441 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.430469 20054 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.431128 19442 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 352c91b410644a06914dd52b1ba5c584 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:39.431308 19558 raft_consensus.cc:2955] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Committing config change with OpId 4.32: config changed from index 31 to 32, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.431427 19452 catalog_manager.cc:5649] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 31 to 32, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New cstate: current_term: 4 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:39.432299 19618 tablet_service.cc:1552] Processing DeleteTablet for tablet 352c91b410644a06914dd52b1ba5c584 with delete_type TABLET_DATA_TOMBSTONED (TS 6b99aaed3fec4c28bf3e3dbbd676ab65 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:39.432502 20457 tablet_replica.cc:333] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:11:39.432564 20457 raft_consensus.cc:2243] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:39.432624 20457 raft_consensus.cc:2272] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:39.432943 20457 ts_tablet_manager.cc:1916] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:39.434176 20457 ts_tablet_manager.cc:1929] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.31
I20251028 21:11:39.434314 20457 log.cc:1199] T 352c91b410644a06914dd52b1ba5c584 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/352c91b410644a06914dd52b1ba5c584
I20251028 21:11:39.434787 19445 catalog_manager.cc:4985] TS 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457): tablet 352c91b410644a06914dd52b1ba5c584 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:39.449666 20054 raft_consensus.cc:1217] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.30->[2.31-2.31] Dedup: 2.31->[]
I20251028 21:11:39.492789 20419 raft_consensus.cc:1064] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1: attempting to promote NON_VOTER a39fb1f187c544f2b77164040db04caf to VOTER
I20251028 21:11:39.492937 20419 consensus_queue.cc:237] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 4.29, Last appended by leader: 28, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:39.493288 20129 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 4 LEARNER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251028 21:11:39.493391 19558 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251028 21:11:39.493309 19631 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 29. Preceding OpId from leader: term: 4 index: 30. (index mismatch)
I20251028 21:11:39.493512 20441 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:39.493609 20419 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:39.493681 20453 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:39.494551 20453 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Committing config change with OpId 4.30: config changed from index 28 to 30, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.494657 19558 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Committing config change with OpId 4.30: config changed from index 28 to 30, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.495437 20129 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 4 FOLLOWER]: Committing config change with OpId 4.30: config changed from index 28 to 30, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.495569 19451 catalog_manager.cc:5649] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 28 to 30, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:39.495672 19631 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Committing config change with OpId 4.30: config changed from index 28 to 30, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.496439 19707 consensus_queue.cc:237] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 4.30, Last appended by leader: 28, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:39.496833 19631 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251028 21:11:39.496833 20129 raft_consensus.cc:1275] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 4 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 30. Preceding OpId from leader: term: 4 index: 31. (index mismatch)
I20251028 21:11:39.496984 20419 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.497074 20441 consensus_queue.cc:1048] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.497555 20311 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Committing config change with OpId 4.31: config changed from index 30 to 31, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.497654 20128 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 30 to 31, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.498422 19442 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 590f65c9d19a4688b25734528ac7474f with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:39.498430 19452 catalog_manager.cc:5649] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf reported cstate change: config changed from index 30 to 31, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New cstate: current_term: 4 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.498700 19633 raft_consensus.cc:2955] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Committing config change with OpId 4.31: config changed from index 30 to 31, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.499138 19546 tablet_service.cc:1552] Processing DeleteTablet for tablet 590f65c9d19a4688b25734528ac7474f with delete_type TABLET_DATA_TOMBSTONED (TS 95350159593b4057ae155343f06246d0 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:53798
I20251028 21:11:39.499325 20461 tablet_replica.cc:333] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:11:39.499413 20461 raft_consensus.cc:2243] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:39.499662 20461 raft_consensus.cc:2272] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:39.499868 20461 ts_tablet_manager.cc:1916] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:39.500979 20461 ts_tablet_manager.cc:1929] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.30
I20251028 21:11:39.501044 20461 log.cc:1199] T 590f65c9d19a4688b25734528ac7474f P 95350159593b4057ae155343f06246d0: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/590f65c9d19a4688b25734528ac7474f
I20251028 21:11:39.501391 19443 catalog_manager.cc:4985] TS 95350159593b4057ae155343f06246d0 (127.15.205.1:38429): tablet 590f65c9d19a4688b25734528ac7474f (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:39.517490 20463 raft_consensus.cc:1064] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: attempting to promote NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 to VOTER
I20251028 21:11:39.517647 20463 consensus_queue.cc:237] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 25, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:39.518108 20054 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251028 21:11:39.518162 19558 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251028 21:11:39.518241 20439 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:39.518334 20462 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:39.518144 19707 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251028 21:11:39.518507 20439 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:39.519129 20462 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Committing config change with OpId 2.28: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.519459 19707 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.520128 20054 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.520128 19558 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.520376 19452 catalog_manager.cc:5649] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index 27 to 28, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:39.521194 19631 consensus_queue.cc:237] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 25, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:39.521543 20054 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251028 21:11:39.521592 19558 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251028 21:11:39.521780 20463 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:39.521878 20439 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:39.522356 20439 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Committing config change with OpId 2.29: config changed from index 28 to 29, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.522449 19558 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.523171 19445 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f with cas_config_opid_index 28: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:39.523238 19451 catalog_manager.cc:5649] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index 28 to 29, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:39.523411 20054 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.524083 19700 tablet_service.cc:1552] Processing DeleteTablet for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f with delete_type TABLET_DATA_TOMBSTONED (TS ac203379ff4e4fdbac249bcd15cf5be1 not found in new config with opid_index 29) from {username='slave'} at 127.0.0.1:41418
I20251028 21:11:39.524224 20468 tablet_replica.cc:333] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:11:39.524317 20468 raft_consensus.cc:2243] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:39.524471 20468 raft_consensus.cc:2272] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:39.524698 20468 ts_tablet_manager.cc:1916] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:39.526086 20468 ts_tablet_manager.cc:1929] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.28
I20251028 21:11:39.526165 20468 log.cc:1199] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P ac203379ff4e4fdbac249bcd15cf5be1: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/8798d7cdf7864b0bbd9e5b2e26d0dc4f
I20251028 21:11:39.526449 19442 catalog_manager.cc:4985] TS ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465): tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:39.588116 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:39.588454 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:39.588521 19707 raft_consensus.cc:606] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Received request to transfer leadership to TS 6b99aaed3fec4c28bf3e3dbbd676ab65
I20251028 21:11:39.588735 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e684839cc93d4e4685039496fbd01e03"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:39.588799 19707 raft_consensus.cc:606] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Received request to transfer leadership to TS 05d39c90d87d4cca9a23f40ad31b12e0
I20251028 21:11:39.589038 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:39.589087 19707 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Received request to transfer leadership to TS 95350159593b4057ae155343f06246d0
I20251028 21:11:39.589208 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251028 21:11:39.589267 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:39.764216 20469 raft_consensus.cc:1064] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: attempting to promote NON_VOTER a39fb1f187c544f2b77164040db04caf to VOTER
I20251028 21:11:39.764405 20469 consensus_queue.cc:237] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 28, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:39.764842 20128 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 LEARNER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:39.764848 19631 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:39.764860 19559 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:39.765070 20471 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.765230 20452 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.765369 20469 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.766050 20452 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.766281 19559 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.767094 19451 catalog_manager.cc:5649] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.767241 20128 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.767354 19631 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.768261 20054 consensus_queue.cc:237] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 28, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:39.768639 20129 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:39.768746 19631 raft_consensus.cc:1275] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:39.768898 20471 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:39.768997 20469 consensus_queue.cc:1048] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:39.769488 20471 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.769636 20129 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.770273 19502 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet ff26670b4bcc4167867fa54b97417365 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:39.770310 19631 raft_consensus.cc:2955] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:39.770385 19451 catalog_manager.cc:5649] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: config changed from index 31 to 32, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New cstate: current_term: 2 leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:39.770911 19546 tablet_service.cc:1552] Processing DeleteTablet for tablet ff26670b4bcc4167867fa54b97417365 with delete_type TABLET_DATA_TOMBSTONED (TS 95350159593b4057ae155343f06246d0 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:53798
I20251028 21:11:39.771016 20461 tablet_replica.cc:333] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:11:39.771070 20461 raft_consensus.cc:2243] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:39.771152 20461 raft_consensus.cc:2272] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:39.771659 20461 ts_tablet_manager.cc:1916] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:39.772845 20461 ts_tablet_manager.cc:1929] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.31
I20251028 21:11:39.772909 20461 log.cc:1199] T ff26670b4bcc4167867fa54b97417365 P 95350159593b4057ae155343f06246d0: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/ff26670b4bcc4167867fa54b97417365
I20251028 21:11:39.773226 19443 catalog_manager.cc:4985] TS 95350159593b4057ae155343f06246d0 (127.15.205.1:38429): tablet ff26670b4bcc4167867fa54b97417365 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:39.835635 20475 raft_consensus.cc:1064] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: attempting to promote NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 to VOTER
I20251028 21:11:39.835860 20475 consensus_queue.cc:237] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 29, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:39.836319 19558 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:39.836319 19707 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:39.836346 20054 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:39.836516 20477 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:39.836707 20478 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:39.836815 20475 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:39.837530 20475 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.837653 19559 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.838307 19707 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.838533 19452 catalog_manager.cc:5649] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a39fb1f187c544f2b77164040db04caf" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.838824 20054 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.839970 20129 consensus_queue.cc:237] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:39.840294 20054 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251028 21:11:39.840471 19559 raft_consensus.cc:1275] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer a39fb1f187c544f2b77164040db04caf: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251028 21:11:39.840541 20477 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:39.840662 20428 consensus_queue.cc:1048] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:39.841204 20428 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.841509 19559 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.841521 20054 raft_consensus.cc:2955] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.842159 19501 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 4c67b4b90e724c8492a4c82e84487f23 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:39.842468 19452 catalog_manager.cc:5649] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: config changed from index 32 to 33, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New cstate: current_term: 2 leader_uuid: "a39fb1f187c544f2b77164040db04caf" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:39.843076 19700 tablet_service.cc:1552] Processing DeleteTablet for tablet 4c67b4b90e724c8492a4c82e84487f23 with delete_type TABLET_DATA_TOMBSTONED (TS ac203379ff4e4fdbac249bcd15cf5be1 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:41418
I20251028 21:11:39.843185 20468 tablet_replica.cc:333] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:11:39.843256 20468 raft_consensus.cc:2243] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:39.843369 20468 raft_consensus.cc:2272] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:39.843650 20468 ts_tablet_manager.cc:1916] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:39.845016 20468 ts_tablet_manager.cc:1929] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.32
I20251028 21:11:39.845074 20468 log.cc:1199] T 4c67b4b90e724c8492a4c82e84487f23 P ac203379ff4e4fdbac249bcd15cf5be1: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/4c67b4b90e724c8492a4c82e84487f23
I20251028 21:11:39.845379 19442 catalog_manager.cc:4985] TS ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465): tablet 4c67b4b90e724c8492a4c82e84487f23 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:39.869868 20441 raft_consensus.cc:993] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:39.869928 20441 raft_consensus.cc:1081] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Signalling peer 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:39.870076 19631 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:37080
I20251028 21:11:39.870131 19631 raft_consensus.cc:493] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:39.870165 19631 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:39.870731 19631 raft_consensus.cc:515] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:39.870874 19631 leader_election.cc:290] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 2 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:39.871181 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:39.871255 19707 raft_consensus.cc:3055] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:39.871305 19707 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:39.871353 19707 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:39.871426 19707 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:39.872083 19707 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 2.
I20251028 21:11:39.872253 19600 leader_election.cc:304] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:39.872337 20439 raft_consensus.cc:2804] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:39.872377 20439 raft_consensus.cc:697] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:39.872414 20439 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:39.872830 19449 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 1 to 2, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:39.875020 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:39.875074 19978 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:39.875535 19978 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 2.
I20251028 21:11:39.942586 20354 raft_consensus.cc:1064] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: attempting to promote NON_VOTER 9e69ab942ba042b6b83938c67405973c to VOTER
I20251028 21:11:39.942736 20354 consensus_queue.cc:237] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 3.29, Last appended by leader: 27, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:39.943094 19978 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 LEARNER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251028 21:11:39.943152 19707 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251028 21:11:39.943218 19631 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251028 21:11:39.943310 20354 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:39.943428 20488 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:39.943498 20354 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:39.944170 20487 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Committing config change with OpId 3.30: config changed from index 29 to 30, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:39.944237 19707 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:39.945050 19978 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:39.945137 19449 catalog_manager.cc:5649] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: config changed from index 29 to 30, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:39.945924 19558 consensus_queue.cc:237] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 27, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:39.945921 19631 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Committing config change with OpId 3.30: config changed from index 29 to 30, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:39.947002 19978 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251028 21:11:39.947042 19707 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251028 21:11:39.947222 20487 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.947361 20485 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:39.947849 20488 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:39.947985 19707 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:39.948738 19443 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 903511324c64491b827a522851cb2402 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:39.948738 19978 raft_consensus.cc:2955] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 30 to 31, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:39.948782 19452 catalog_manager.cc:5649] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 30 to 31, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New cstate: current_term: 3 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:39.949419 19618 tablet_service.cc:1552] Processing DeleteTablet for tablet 903511324c64491b827a522851cb2402 with delete_type TABLET_DATA_TOMBSTONED (TS 6b99aaed3fec4c28bf3e3dbbd676ab65 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:39.949631 20492 tablet_replica.cc:333] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:11:39.949685 20492 raft_consensus.cc:2243] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:39.949764 20492 raft_consensus.cc:2272] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:39.949939 20492 ts_tablet_manager.cc:1916] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:39.950943 20492 ts_tablet_manager.cc:1929] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.30
I20251028 21:11:39.951014 20492 log.cc:1199] T 903511324c64491b827a522851cb2402 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/903511324c64491b827a522851cb2402
I20251028 21:11:39.951326 19445 catalog_manager.cc:4985] TS 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457): tablet 903511324c64491b827a522851cb2402 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:39.963917 20441 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:39.963977 20441 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Signalling peer 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:39.964118 19559 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:53816
I20251028 21:11:39.964169 19559 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:39.964187 19559 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:39.964639 19559 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:39.964733 19559 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 5 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:39.964864 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:39.964923 19707 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Stepping down as leader of term 4
I20251028 21:11:39.964908 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:39.964984 19707 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:39.965008 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:39.965030 19707 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:39.965121 19707 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:39.965507 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 5.
I20251028 21:11:39.965564 19707 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 5.
I20251028 21:11:39.965668 19524 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, 9e69ab942ba042b6b83938c67405973c; no voters:
I20251028 21:11:39.965736 20487 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 5 FOLLOWER]: Leader election won for term 5
I20251028 21:11:39.965792 20487 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 5 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:39.965880 20487 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:39.966320 19449 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 4 to 5, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 5 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:40.041106 20441 raft_consensus.cc:993] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 05d39c90d87d4cca9a23f40ad31b12e0 to start an election
I20251028 21:11:40.041173 20441 raft_consensus.cc:1081] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Signalling peer 05d39c90d87d4cca9a23f40ad31b12e0 to start an election
I20251028 21:11:40.041326 20054 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e684839cc93d4e4685039496fbd01e03"
dest_uuid: "05d39c90d87d4cca9a23f40ad31b12e0"
from {username='slave'} at 127.0.0.1:42776
I20251028 21:11:40.041395 20054 raft_consensus.cc:493] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:40.041429 20054 raft_consensus.cc:3060] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:40.041955 20054 raft_consensus.cc:515] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:40.042088 20054 leader_election.cc:290] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [CANDIDATE]: Term 2 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471)
I20251028 21:11:40.042235 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e684839cc93d4e4685039496fbd01e03" candidate_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:40.042251 20128 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e684839cc93d4e4685039496fbd01e03" candidate_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "a39fb1f187c544f2b77164040db04caf"
I20251028 21:11:40.042304 19707 raft_consensus.cc:3055] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:11:40.042330 19707 raft_consensus.cc:740] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:40.042320 20128 raft_consensus.cc:3060] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:40.042420 19707 consensus_queue.cc:260] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:40.042516 19707 raft_consensus.cc:3060] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:11:40.042888 20128 raft_consensus.cc:2468] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d39c90d87d4cca9a23f40ad31b12e0 in term 2.
I20251028 21:11:40.042958 19707 raft_consensus.cc:2468] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d39c90d87d4cca9a23f40ad31b12e0 in term 2.
I20251028 21:11:40.043035 20017 leader_election.cc:304] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d39c90d87d4cca9a23f40ad31b12e0, a39fb1f187c544f2b77164040db04caf; no voters:
I20251028 21:11:40.043107 20452 raft_consensus.cc:2804] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:11:40.043154 20452 raft_consensus.cc:697] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Becoming Leader. State: Replica: 05d39c90d87d4cca9a23f40ad31b12e0, State: Running, Role: LEADER
I20251028 21:11:40.043200 20452 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:40.043632 19449 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: term changed from 1 to 2, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5). New cstate: current_term: 2 leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:40.271715 19707 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:40.271942 20463 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:40.273056 19978 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:40.273224 20497 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:40.390892 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 5 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251028 21:11:40.391124 20487 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:40.392105 19707 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251028 21:11:40.392273 20485 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:40.506428 19707 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251028 21:11:40.506630 20452 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:40.507615 20128 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251028 21:11:40.507786 20506 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:40.950482 20054 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:40.951107 19707 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251028 21:11:40.951121 20128 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251028 21:11:40.951354 20417 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251028 21:11:40.951429 20452 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251028 21:11:40.952062 20506 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:40.953191 19449 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: config changed from index 32 to 34, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New cstate: current_term: 2 leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:40.953233 20128 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:40.954099 19559 consensus_queue.cc:237] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 2.27, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:40.954555 19631 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251028 21:11:40.954594 19712 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
W20251028 21:11:40.954659 19524 consensus_peers.cc:597] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 -> Peer 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831): Couldn't send request to peer 9e69ab942ba042b6b83938c67405973c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10b6448e96bb4fd08006249ba36d650e. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:40.954732 20510 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:40.953202 19707 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:40.954924 20516 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251028 21:11:40.955511 20485 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
W20251028 21:11:40.955782 20017 consensus_peers.cc:597] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 -> Peer 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831): Couldn't send request to peer 9e69ab942ba042b6b83938c67405973c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e684839cc93d4e4685039496fbd01e03. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:40.955732 19631 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:40.955871 19707 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.28: config changed from index -1 to 28, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } } }
I20251028 21:11:40.956605 19450 catalog_manager.cc:5649] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index -1 to 28, NON_VOTER 9e69ab942ba042b6b83938c67405973c (127.15.205.4) added. New cstate: current_term: 2 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:40.956867 19631 consensus_queue.cc:237] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 25, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
W20251028 21:11:40.957334 19600 consensus_peers.cc:597] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 -> Peer a39fb1f187c544f2b77164040db04caf (127.15.205.6:45471): Couldn't send request to peer a39fb1f187c544f2b77164040db04caf. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8798d7cdf7864b0bbd9e5b2e26d0dc4f. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:40.957461 19559 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:40.957459 20054 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:40.957670 20521 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:40.957765 20497 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:40.958360 20497 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index 29 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:40.958504 19559 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 29 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:40.959292 20054 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 29 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } } }
I20251028 21:11:40.959328 19453 catalog_manager.cc:5649] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index 29 to 30, NON_VOTER a39fb1f187c544f2b77164040db04caf (127.15.205.6) added. New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:40.959780 19977 consensus_queue.cc:237] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:40.960258 19631 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:40.960608 20525 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:40.960739 19707 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:40.960880 20525 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:40.961606 20293 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:40.961740 19707 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:40.962447 19631 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 29 to 31, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } } }
I20251028 21:11:40.962630 19450 catalog_manager.cc:5649] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c reported cstate change: config changed from index 29 to 31, NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) added. New cstate: current_term: 2 leader_uuid: "9e69ab942ba042b6b83938c67405973c" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251028 21:11:40.963583 19942 consensus_peers.cc:597] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c -> Peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477): Couldn't send request to peer 05d39c90d87d4cca9a23f40ad31b12e0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f49b5a0ce6fe4d91aa0ab60a1d770949. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:11:41.374348 20531 ts_tablet_manager.cc:933] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: Initiating tablet copy from peer 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:41.374529 20531 tablet_copy_client.cc:323] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.4:42831
I20251028 21:11:41.374711 19988 tablet_copy_service.cc:140] P 9e69ab942ba042b6b83938c67405973c: Received BeginTabletCopySession request for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 from peer 05d39c90d87d4cca9a23f40ad31b12e0 ({username='slave'} at 127.0.0.1:44006)
I20251028 21:11:41.374779 19988 tablet_copy_service.cc:161] P 9e69ab942ba042b6b83938c67405973c: Beginning new tablet copy session on tablet f49b5a0ce6fe4d91aa0ab60a1d770949 from peer 05d39c90d87d4cca9a23f40ad31b12e0 at {username='slave'} at 127.0.0.1:44006: session id = 05d39c90d87d4cca9a23f40ad31b12e0-f49b5a0ce6fe4d91aa0ab60a1d770949
I20251028 21:11:41.375221 19988 tablet_copy_source_session.cc:215] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:41.375563 20531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49b5a0ce6fe4d91aa0ab60a1d770949. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:41.376927 20531 tablet_copy_client.cc:806] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:41.377095 20531 tablet_copy_client.cc:670] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:41.377585 20532 tablet_copy_client.cc:1025] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:41.486541 20533 ts_tablet_manager.cc:933] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: Initiating tablet copy from peer 95350159593b4057ae155343f06246d0 (127.15.205.1:38429)
I20251028 21:11:41.486696 20533 tablet_copy_client.cc:323] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.1:38429
I20251028 21:11:41.486927 19571 tablet_copy_service.cc:140] P 95350159593b4057ae155343f06246d0: Received BeginTabletCopySession request for tablet 10b6448e96bb4fd08006249ba36d650e from peer 9e69ab942ba042b6b83938c67405973c ({username='slave'} at 127.0.0.1:53846)
I20251028 21:11:41.486982 19571 tablet_copy_service.cc:161] P 95350159593b4057ae155343f06246d0: Beginning new tablet copy session on tablet 10b6448e96bb4fd08006249ba36d650e from peer 9e69ab942ba042b6b83938c67405973c at {username='slave'} at 127.0.0.1:53846: session id = 9e69ab942ba042b6b83938c67405973c-10b6448e96bb4fd08006249ba36d650e
I20251028 21:11:41.487566 19571 tablet_copy_source_session.cc:215] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:41.487854 20533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10b6448e96bb4fd08006249ba36d650e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:41.489027 20533 tablet_copy_client.cc:806] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:41.489140 20533 tablet_copy_client.cc:670] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:41.489601 20534 tablet_copy_client.cc:1025] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:41.535218 20535 ts_tablet_manager.cc:933] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: Initiating tablet copy from peer 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457)
I20251028 21:11:41.535365 20535 tablet_copy_client.cc:323] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:36457
I20251028 21:11:41.535534 19647 tablet_copy_service.cc:140] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Received BeginTabletCopySession request for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f from peer a39fb1f187c544f2b77164040db04caf ({username='slave'} at 127.0.0.1:53888)
I20251028 21:11:41.535580 19647 tablet_copy_service.cc:161] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Beginning new tablet copy session on tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f from peer a39fb1f187c544f2b77164040db04caf at {username='slave'} at 127.0.0.1:53888: session id = a39fb1f187c544f2b77164040db04caf-8798d7cdf7864b0bbd9e5b2e26d0dc4f
I20251028 21:11:41.536110 19647 tablet_copy_source_session.cc:215] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:41.536548 20535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:41.537729 20535 tablet_copy_client.cc:806] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:41.537895 20535 tablet_copy_client.cc:670] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:41.538340 20536 tablet_copy_client.cc:1025] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:41.557650 20537 ts_tablet_manager.cc:933] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: Initiating tablet copy from peer 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477)
I20251028 21:11:41.557825 20537 tablet_copy_client.cc:323] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.5:44477
I20251028 21:11:41.557997 20064 tablet_copy_service.cc:140] P 05d39c90d87d4cca9a23f40ad31b12e0: Received BeginTabletCopySession request for tablet e684839cc93d4e4685039496fbd01e03 from peer 9e69ab942ba042b6b83938c67405973c ({username='slave'} at 127.0.0.1:47518)
I20251028 21:11:41.558055 20064 tablet_copy_service.cc:161] P 05d39c90d87d4cca9a23f40ad31b12e0: Beginning new tablet copy session on tablet e684839cc93d4e4685039496fbd01e03 from peer 9e69ab942ba042b6b83938c67405973c at {username='slave'} at 127.0.0.1:47518: session id = 9e69ab942ba042b6b83938c67405973c-e684839cc93d4e4685039496fbd01e03
I20251028 21:11:41.558581 20064 tablet_copy_source_session.cc:215] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:11:41.558880 20537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e684839cc93d4e4685039496fbd01e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:11:41.560204 20537 tablet_copy_client.cc:806] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 0 data blocks...
I20251028 21:11:41.560345 20537 tablet_copy_client.cc:670] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Starting download of 1 WAL segments...
I20251028 21:11:41.560786 20538 tablet_copy_client.cc:1025] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Injecting latency into file download: 2000
I20251028 21:11:41.589370 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:41.589696 19559 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "903511324c64491b827a522851cb2402"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "9e69ab942ba042b6b83938c67405973c"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:41.589772 19559 raft_consensus.cc:606] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Received request to transfer leadership to TS 9e69ab942ba042b6b83938c67405973c
I20251028 21:11:41.590045 19631 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:37102
I20251028 21:11:41.590109 19631 raft_consensus.cc:606] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:41.590240 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251028 21:11:41.590296 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:41.940485 20539 raft_consensus.cc:993] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: : Instructing follower 9e69ab942ba042b6b83938c67405973c to start an election
I20251028 21:11:41.940575 20539 raft_consensus.cc:1081] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Signalling peer 9e69ab942ba042b6b83938c67405973c to start an election
I20251028 21:11:41.940764 19977 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "903511324c64491b827a522851cb2402"
dest_uuid: "9e69ab942ba042b6b83938c67405973c"
from {username='slave'} at 127.0.0.1:60432
I20251028 21:11:41.940893 19977 raft_consensus.cc:493] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:41.940915 19977 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:41.941617 19977 raft_consensus.cc:515] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:41.941788 19977 leader_election.cc:290] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [CANDIDATE]: Term 4 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465)
I20251028 21:11:41.941865 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "9e69ab942ba042b6b83938c67405973c" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:41.941888 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "903511324c64491b827a522851cb2402" candidate_uuid: "9e69ab942ba042b6b83938c67405973c" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:41.941957 19558 raft_consensus.cc:3055] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Stepping down as leader of term 3
I20251028 21:11:41.941970 19707 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:41.942000 19558 raft_consensus.cc:740] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:41.942045 19558 consensus_queue.cc:260] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:41.942098 19558 raft_consensus.cc:3060] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:41.942485 19707 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e69ab942ba042b6b83938c67405973c in term 4.
I20251028 21:11:41.942618 19558 raft_consensus.cc:2468] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e69ab942ba042b6b83938c67405973c in term 4.
I20251028 21:11:41.942770 19941 leader_election.cc:304] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e69ab942ba042b6b83938c67405973c, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:41.942885 20540 raft_consensus.cc:2804] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Leader election won for term 4
I20251028 21:11:41.943032 20540 raft_consensus.cc:697] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 4 LEADER]: Becoming Leader. State: Replica: 9e69ab942ba042b6b83938c67405973c, State: Running, Role: LEADER
I20251028 21:11:41.943113 20540 consensus_queue.cc:237] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:41.943563 19453 catalog_manager.cc:5649] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c reported cstate change: term changed from 3 to 4, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to 9e69ab942ba042b6b83938c67405973c (127.15.205.4). New cstate: current_term: 4 leader_uuid: "9e69ab942ba042b6b83938c67405973c" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:41.988206 20546 raft_consensus.cc:993] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:41.988266 20546 raft_consensus.cc:1081] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:41.988416 19707 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41426
I20251028 21:11:41.988492 19707 raft_consensus.cc:493] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:41.988517 19707 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:41.989233 19707 raft_consensus.cc:515] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:41.989387 19707 leader_election.cc:290] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 3 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:41.989473 19631 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:41.989524 19977 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:41.989581 19977 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:41.989544 19631 raft_consensus.cc:3055] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:11:41.989665 19631 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:41.989727 19631 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:41.989794 19631 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:11:41.990162 19977 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 3.
I20251028 21:11:41.990283 19631 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 3.
I20251028 21:11:41.990306 19675 leader_election.cc:304] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e69ab942ba042b6b83938c67405973c, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:41.990414 20441 raft_consensus.cc:2804] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:11:41.990499 20441 raft_consensus.cc:697] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:41.990566 20441 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:41.991027 19453 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 2 to 3, leader changed from 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 3 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:42.359328 19558 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251028 21:11:42.359539 20541 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:42.360706 19707 raft_consensus.cc:1275] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251028 21:11:42.360946 20540 consensus_queue.cc:1048] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:42.521929 19977 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251028 21:11:42.522135 20441 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:42.523123 19631 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251028 21:11:42.523366 20441 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:43.377764 20531 tablet_copy_client.cc:538] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:43.380082 20531 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap starting.
I20251028 21:11:43.384545 20531 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=114 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:43.384730 20531 tablet_bootstrap.cc:492] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: Bootstrap complete.
I20251028 21:11:43.384835 20531 ts_tablet_manager.cc:1403] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251028 21:11:43.384984 20531 raft_consensus.cc:359] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:43.385062 20531 raft_consensus.cc:740] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05d39c90d87d4cca9a23f40ad31b12e0, State: Initialized, Role: LEARNER
I20251028 21:11:43.385133 20531 consensus_queue.cc:260] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: NON_VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: true } }
I20251028 21:11:43.385229 20531 ts_tablet_manager.cc:1434] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.002s
I20251028 21:11:43.385394 19988 tablet_copy_service.cc:342] P 9e69ab942ba042b6b83938c67405973c: Request end of tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-f49b5a0ce6fe4d91aa0ab60a1d770949 received from {username='slave'} at 127.0.0.1:44006
I20251028 21:11:43.385458 19988 tablet_copy_service.cc:434] P 9e69ab942ba042b6b83938c67405973c: ending tablet copy session 05d39c90d87d4cca9a23f40ad31b12e0-f49b5a0ce6fe4d91aa0ab60a1d770949 on tablet f49b5a0ce6fe4d91aa0ab60a1d770949 with peer 05d39c90d87d4cca9a23f40ad31b12e0
I20251028 21:11:43.489796 20533 tablet_copy_client.cc:538] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:43.490764 20533 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: Bootstrap starting.
I20251028 21:11:43.494402 20533 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: Bootstrap replayed 1/1 log segments. Stats: ops{read=28 overwritten=0 applied=28 ignored=0} inserts{seen=101 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:43.494582 20533 tablet_bootstrap.cc:492] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: Bootstrap complete.
I20251028 21:11:43.494675 20533 ts_tablet_manager.cc:1403] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 21:11:43.494818 20533 raft_consensus.cc:359] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:43.494899 20533 raft_consensus.cc:740] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 9e69ab942ba042b6b83938c67405973c, State: Initialized, Role: LEARNER
I20251028 21:11:43.494941 20533 consensus_queue.cc:260] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 28, Last appended: 2.28, Last appended by leader: 28, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:43.495028 20533 ts_tablet_manager.cc:1434] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:43.495187 19571 tablet_copy_service.cc:342] P 95350159593b4057ae155343f06246d0: Request end of tablet copy session 9e69ab942ba042b6b83938c67405973c-10b6448e96bb4fd08006249ba36d650e received from {username='slave'} at 127.0.0.1:53846
I20251028 21:11:43.495245 19571 tablet_copy_service.cc:434] P 95350159593b4057ae155343f06246d0: ending tablet copy session 9e69ab942ba042b6b83938c67405973c-10b6448e96bb4fd08006249ba36d650e on tablet 10b6448e96bb4fd08006249ba36d650e with peer 9e69ab942ba042b6b83938c67405973c
I20251028 21:11:43.538522 20535 tablet_copy_client.cc:538] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:43.539373 20535 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: Bootstrap starting.
I20251028 21:11:43.543315 20535 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=111 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:43.543498 20535 tablet_bootstrap.cc:492] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: Bootstrap complete.
I20251028 21:11:43.543593 20535 ts_tablet_manager.cc:1403] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20251028 21:11:43.543726 20535 raft_consensus.cc:359] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:43.543781 20535 raft_consensus.cc:740] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a39fb1f187c544f2b77164040db04caf, State: Initialized, Role: LEARNER
I20251028 21:11:43.543812 20535 consensus_queue.cc:260] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: NON_VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: true } }
I20251028 21:11:43.543906 20535 ts_tablet_manager.cc:1434] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:43.544108 19647 tablet_copy_service.cc:342] P 6b99aaed3fec4c28bf3e3dbbd676ab65: Request end of tablet copy session a39fb1f187c544f2b77164040db04caf-8798d7cdf7864b0bbd9e5b2e26d0dc4f received from {username='slave'} at 127.0.0.1:53888
I20251028 21:11:43.544168 19647 tablet_copy_service.cc:434] P 6b99aaed3fec4c28bf3e3dbbd676ab65: ending tablet copy session a39fb1f187c544f2b77164040db04caf-8798d7cdf7864b0bbd9e5b2e26d0dc4f on tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f with peer a39fb1f187c544f2b77164040db04caf
I20251028 21:11:43.560961 20537 tablet_copy_client.cc:538] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:11:43.561627 20537 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: Bootstrap starting.
I20251028 21:11:43.565837 20537 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=34 ignored=0} inserts{seen=93 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:11:43.566047 20537 tablet_bootstrap.cc:492] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: Bootstrap complete.
I20251028 21:11:43.566164 20537 ts_tablet_manager.cc:1403] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 21:11:43.566309 20537 raft_consensus.cc:359] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:43.566385 20537 raft_consensus.cc:740] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 9e69ab942ba042b6b83938c67405973c, State: Initialized, Role: LEARNER
I20251028 21:11:43.566419 20537 consensus_queue.cc:260] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 34, Last appended: 2.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: true } }
I20251028 21:11:43.566519 20537 ts_tablet_manager.cc:1434] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:11:43.566679 20064 tablet_copy_service.cc:342] P 05d39c90d87d4cca9a23f40ad31b12e0: Request end of tablet copy session 9e69ab942ba042b6b83938c67405973c-e684839cc93d4e4685039496fbd01e03 received from {username='slave'} at 127.0.0.1:47518
I20251028 21:11:43.566730 20064 tablet_copy_service.cc:434] P 05d39c90d87d4cca9a23f40ad31b12e0: ending tablet copy session 9e69ab942ba042b6b83938c67405973c-e684839cc93d4e4685039496fbd01e03 on tablet e684839cc93d4e4685039496fbd01e03 with peer 9e69ab942ba042b6b83938c67405973c
I20251028 21:11:43.590435 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:43.590806 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:43.590871 19707 raft_consensus.cc:606] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Received request to transfer leadership to TS 95350159593b4057ae155343f06246d0
I20251028 21:11:43.591048 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:43.591122 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:43.756769 20565 raft_consensus.cc:993] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:43.756852 20565 raft_consensus.cc:1081] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Signalling peer 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:43.757091 19558 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584"
dest_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:53816
I20251028 21:11:43.757164 19558 raft_consensus.cc:493] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:43.757191 19558 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:43.757908 19558 raft_consensus.cc:515] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:43.758071 19558 leader_election.cc:290] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 5 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5:44477)
I20251028 21:11:43.758188 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:43.758203 20053 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "352c91b410644a06914dd52b1ba5c584" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "05d39c90d87d4cca9a23f40ad31b12e0"
I20251028 21:11:43.758278 19707 raft_consensus.cc:3055] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Stepping down as leader of term 4
I20251028 21:11:43.758306 19707 raft_consensus.cc:740] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:43.758307 20053 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:43.758365 19707 consensus_queue.cc:260] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:43.758455 19707 raft_consensus.cc:3060] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:43.758872 20053 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 5.
I20251028 21:11:43.759002 19707 raft_consensus.cc:2468] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 5.
I20251028 21:11:43.759021 19525 leader_election.cc:304] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d39c90d87d4cca9a23f40ad31b12e0, 95350159593b4057ae155343f06246d0; no voters:
I20251028 21:11:43.759099 20556 raft_consensus.cc:2804] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 5 FOLLOWER]: Leader election won for term 5
I20251028 21:11:43.759207 20556 raft_consensus.cc:697] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 5 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:43.759310 20556 consensus_queue.cc:237] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:43.759867 19451 catalog_manager.cc:5649] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 4 to 5, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 5 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:43.781993 20053 raft_consensus.cc:1217] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.30->[2.31-2.31] Dedup: 2.31->[]
I20251028 21:11:44.036070 19977 raft_consensus.cc:1217] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.33->[2.34-2.34] Dedup: 2.34->[]
I20251028 21:11:44.037573 20128 raft_consensus.cc:1217] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 LEARNER]: Deduplicated request from leader. Original: 2.29->[2.30-2.30] Dedup: 2.30->[]
I20251028 21:11:44.057201 19977 raft_consensus.cc:1217] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.27->[2.28-2.28] Dedup: 2.28->[]
I20251028 21:11:44.139465 20054 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 5 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251028 21:11:44.139639 20556 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:44.140781 19707 raft_consensus.cc:1275] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251028 21:11:44.141036 20580 consensus_queue.cc:1048] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:44.191901 20585 raft_consensus.cc:1064] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: attempting to promote NON_VOTER 05d39c90d87d4cca9a23f40ad31b12e0 to VOTER
I20251028 21:11:44.192117 20585 consensus_queue.cc:237] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 29, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:44.192624 20054 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEARNER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:44.192642 19631 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:44.192658 19707 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:44.192853 20587 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:44.192991 20584 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:44.193130 20585 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:44.193825 19631 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:44.193732 20587 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:44.193903 20054 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:44.194864 19453 catalog_manager.cc:5649] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c reported cstate change: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9e69ab942ba042b6b83938c67405973c" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:44.195173 19707 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 05d39c90d87d4cca9a23f40ad31b12e0 (127.15.205.5) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } attrs { replace: true } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:44.196533 19978 consensus_queue.cc:237] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } }
I20251028 21:11:44.196863 20054 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251028 21:11:44.196913 19707 raft_consensus.cc:1275] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 9e69ab942ba042b6b83938c67405973c: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251028 21:11:44.197008 20587 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:44.197077 20541 consensus_queue.cc:1048] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:44.197559 20541 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:44.197654 20053 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:44.198388 19501 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:44.198437 19707 raft_consensus.cc:2955] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:44.198462 19451 catalog_manager.cc:5649] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: config changed from index 32 to 33, VOTER 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) evicted. New cstate: current_term: 2 leader_uuid: "9e69ab942ba042b6b83938c67405973c" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } }
I20251028 21:11:44.199592 19618 tablet_service.cc:1552] Processing DeleteTablet for tablet f49b5a0ce6fe4d91aa0ab60a1d770949 with delete_type TABLET_DATA_TOMBSTONED (TS 6b99aaed3fec4c28bf3e3dbbd676ab65 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:37054
I20251028 21:11:44.199764 20591 tablet_replica.cc:333] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:11:44.199834 20591 raft_consensus.cc:2243] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:44.199927 20591 raft_consensus.cc:2272] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:44.200138 20591 ts_tablet_manager.cc:1916] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:44.201447 20591 ts_tablet_manager.cc:1929] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.32
I20251028 21:11:44.201503 20591 log.cc:1199] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 6b99aaed3fec4c28bf3e3dbbd676ab65: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/f49b5a0ce6fe4d91aa0ab60a1d770949
I20251028 21:11:44.201824 19445 catalog_manager.cc:4985] TS 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457): tablet f49b5a0ce6fe4d91aa0ab60a1d770949 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:44.491811 20592 raft_consensus.cc:1064] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: attempting to promote NON_VOTER a39fb1f187c544f2b77164040db04caf to VOTER
I20251028 21:11:44.492066 20592 consensus_queue.cc:237] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 25, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:44.492630 20128 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 LEARNER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:44.492722 19558 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:44.492641 20053 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251028 21:11:44.492908 20594 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:44.493018 20566 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:44.493187 20566 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251028 21:11:44.493841 20594 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.494004 20053 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.495038 20128 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.495085 19453 catalog_manager.cc:5649] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.495106 19558 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index 30 to 31, a39fb1f187c544f2b77164040db04caf (127.15.205.6) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } attrs { replace: true } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.496426 19633 consensus_queue.cc:237] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 25, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } }
I20251028 21:11:44.496748 20053 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:44.496748 20128 raft_consensus.cc:1275] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251028 21:11:44.496945 20592 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:44.497025 20566 consensus_queue.cc:1048] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:44.497447 20592 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.497617 20128 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.498144 19445 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:44.498301 19452 catalog_manager.cc:5649] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf reported cstate change: config changed from index 31 to 32, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New cstate: current_term: 2 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.498569 20053 raft_consensus.cc:2955] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, VOTER 95350159593b4057ae155343f06246d0 (127.15.205.1) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } }
I20251028 21:11:44.498883 19546 tablet_service.cc:1552] Processing DeleteTablet for tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f with delete_type TABLET_DATA_TOMBSTONED (TS 95350159593b4057ae155343f06246d0 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:53798
I20251028 21:11:44.499058 20599 tablet_replica.cc:333] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:11:44.499116 20599 raft_consensus.cc:2243] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:44.499189 20599 raft_consensus.cc:2272] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:44.499661 20599 ts_tablet_manager.cc:1916] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:44.500964 20599 ts_tablet_manager.cc:1929] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.31
I20251028 21:11:44.501015 20599 log.cc:1199] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 95350159593b4057ae155343f06246d0: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/8798d7cdf7864b0bbd9e5b2e26d0dc4f
I20251028 21:11:44.501284 19443 catalog_manager.cc:4985] TS 95350159593b4057ae155343f06246d0 (127.15.205.1:38429): tablet 8798d7cdf7864b0bbd9e5b2e26d0dc4f (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:44.512413 20556 raft_consensus.cc:1064] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0: attempting to promote NON_VOTER 9e69ab942ba042b6b83938c67405973c to VOTER
I20251028 21:11:44.512570 20556 consensus_queue.cc:237] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 26, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:44.513060 19707 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251028 21:11:44.513140 19978 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 LEARNER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251028 21:11:44.513214 19633 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251028 21:11:44.513242 20572 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:44.513363 20572 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:44.513554 20572 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251028 21:11:44.514006 20601 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Committing config change with OpId 2.29: config changed from index 28 to 29, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.514079 19707 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.514839 19450 catalog_manager.cc:5649] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 reported cstate change: config changed from index 28 to 29, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:44.514989 19978 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.514989 19633 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index 28 to 29, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.516584 19558 consensus_queue.cc:237] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 26, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:44.516858 19633 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:44.516885 19978 raft_consensus.cc:1275] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251028 21:11:44.517076 20601 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:44.517189 20572 consensus_queue.cc:1048] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251028 21:11:44.517573 20601 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.517660 19978 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.517733 19633 raft_consensus.cc:2955] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Committing config change with OpId 2.30: config changed from index 29 to 30, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.518270 19443 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 10b6448e96bb4fd08006249ba36d650e with cas_config_opid_index 29: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:44.518422 19451 catalog_manager.cc:5649] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: config changed from index 29 to 30, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New cstate: current_term: 2 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.519011 19700 tablet_service.cc:1552] Processing DeleteTablet for tablet 10b6448e96bb4fd08006249ba36d650e with delete_type TABLET_DATA_TOMBSTONED (TS ac203379ff4e4fdbac249bcd15cf5be1 not found in new config with opid_index 30) from {username='slave'} at 127.0.0.1:41418
I20251028 21:11:44.519198 20605 tablet_replica.cc:333] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:11:44.519284 20605 raft_consensus.cc:2243] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:44.519345 20605 raft_consensus.cc:2272] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:44.519579 20605 ts_tablet_manager.cc:1916] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:44.520833 20605 ts_tablet_manager.cc:1929] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.29
I20251028 21:11:44.520888 20605 log.cc:1199] T 10b6448e96bb4fd08006249ba36d650e P ac203379ff4e4fdbac249bcd15cf5be1: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/10b6448e96bb4fd08006249ba36d650e
I20251028 21:11:44.521236 19442 catalog_manager.cc:4985] TS ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465): tablet 10b6448e96bb4fd08006249ba36d650e (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:44.577185 20559 raft_consensus.cc:1064] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: attempting to promote NON_VOTER 9e69ab942ba042b6b83938c67405973c to VOTER
I20251028 21:11:44.577411 20559 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 32, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:44.577924 20128 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251028 21:11:44.578009 19978 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 LEARNER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251028 21:11:44.578047 19707 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251028 21:11:44.578408 20608 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251028 21:11:44.578524 20559 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251028 21:11:44.578702 20606 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251028 21:11:44.579413 20559 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.579514 20128 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.580374 19978 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.580389 19452 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf reported cstate change: config changed from index 34 to 35, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.581483 19707 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 9e69ab942ba042b6b83938c67405973c (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } attrs { replace: true } } peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.582309 20053 consensus_queue.cc:237] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 2.35, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:44.582700 19978 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20251028 21:11:44.582753 20128 raft_consensus.cc:1275] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Refusing update from remote peer 05d39c90d87d4cca9a23f40ad31b12e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20251028 21:11:44.582983 20606 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251028 21:11:44.583096 20608 consensus_queue.cc:1048] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251028 21:11:44.583640 20559 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.584337 19978 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.584398 19502 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet e684839cc93d4e4685039496fbd01e03 with cas_config_opid_index 35: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:11:44.584385 19450 catalog_manager.cc:5649] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 reported cstate change: config changed from index 35 to 36, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New cstate: current_term: 2 leader_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:11:44.584465 20128 raft_consensus.cc:2955] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "05d39c90d87d4cca9a23f40ad31b12e0" member_type: VOTER last_known_addr { host: "127.15.205.5" port: 44477 } attrs { promote: false } } peers { permanent_uuid: "a39fb1f187c544f2b77164040db04caf" member_type: VOTER last_known_addr { host: "127.15.205.6" port: 45471 } attrs { promote: false } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } } }
I20251028 21:11:44.585137 19700 tablet_service.cc:1552] Processing DeleteTablet for tablet e684839cc93d4e4685039496fbd01e03 with delete_type TABLET_DATA_TOMBSTONED (TS ac203379ff4e4fdbac249bcd15cf5be1 not found in new config with opid_index 36) from {username='slave'} at 127.0.0.1:41418
I20251028 21:11:44.585242 20605 tablet_replica.cc:333] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:11:44.585304 20605 raft_consensus.cc:2243] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:11:44.585363 20605 raft_consensus.cc:2272] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:11:44.585665 20605 ts_tablet_manager.cc:1916] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:11:44.586753 20605 ts_tablet_manager.cc:1929] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.35
I20251028 21:11:44.586800 20605 log.cc:1199] T e684839cc93d4e4685039496fbd01e03 P ac203379ff4e4fdbac249bcd15cf5be1: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/e684839cc93d4e4685039496fbd01e03
I20251028 21:11:44.587102 19442 catalog_manager.cc:4985] TS ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465): tablet e684839cc93d4e4685039496fbd01e03 (table test-workload [id=19436255528a4488ac9e29c7f47516fb]) successfully deleted
I20251028 21:11:45.591363 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:45.591768 19558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:45.591854 19558 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 5 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:45.592032 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:45.592108 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:45.849519 20572 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:45.849602 20572 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 5 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:45.849874 19707 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41428
I20251028 21:11:45.849958 19707 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:45.849988 19707 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Advancing to term 6
I20251028 21:11:45.850607 19707 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:45.850785 19707 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 6 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:45.850931 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:45.850975 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:45.850998 19558 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 5 LEADER]: Stepping down as leader of term 5
I20251028 21:11:45.851039 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 5 FOLLOWER]: Advancing to term 6
I20251028 21:11:45.851051 19558 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:45.851116 19558 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:45.851188 19558 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 5 FOLLOWER]: Advancing to term 6
I20251028 21:11:45.851655 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 6.
I20251028 21:11:45.851805 19558 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 6.
I20251028 21:11:45.851856 19675 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e69ab942ba042b6b83938c67405973c, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:45.851953 20571 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 FOLLOWER]: Leader election won for term 6
I20251028 21:11:45.852056 20571 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:45.852108 20571 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:45.852613 19452 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 5 to 6, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 6 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:46.384884 19558 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 6 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 5 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251028 21:11:46.385099 20571 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251028 21:11:46.386247 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 6 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 5 index: 33. Preceding OpId from leader: term: 6 index: 34. (index mismatch)
I20251028 21:11:46.386502 20620 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251028 21:11:47.592237 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:47.592617 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:47.592698 19707 raft_consensus.cc:606] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 LEADER]: Received request to transfer leadership to TS 6b99aaed3fec4c28bf3e3dbbd676ab65
I20251028 21:11:47.592921 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:47.592983 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:47.997277 20620 raft_consensus.cc:993] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:47.997368 20620 raft_consensus.cc:1081] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 LEADER]: Signalling peer 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:11:47.997498 19633 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:37080
I20251028 21:11:47.997562 19633 raft_consensus.cc:493] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:47.997581 19633 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:47.998116 19633 raft_consensus.cc:515] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:47.998245 19633 leader_election.cc:290] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 4 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:47.998397 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:47.998426 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:47.998503 19707 raft_consensus.cc:3055] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 LEADER]: Stepping down as leader of term 3
I20251028 21:11:47.998543 19978 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:47.998551 19707 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:47.998627 19707 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:47.998714 19707 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 3 FOLLOWER]: Advancing to term 4
I20251028 21:11:47.999120 19978 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 4.
I20251028 21:11:47.999311 19600 leader_election.cc:304] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 9e69ab942ba042b6b83938c67405973c; no voters:
I20251028 21:11:47.999337 19707 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 4.
I20251028 21:11:47.999385 20598 raft_consensus.cc:2804] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Leader election won for term 4
I20251028 21:11:47.999480 20598 raft_consensus.cc:697] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:47.999537 20598 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:47.999991 19452 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 3 to 4, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 4 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:48.426198 19707 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251028 21:11:48.426409 20598 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:48.427551 19978 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Refusing update from remote peer 6b99aaed3fec4c28bf3e3dbbd676ab65: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251028 21:11:48.427733 20598 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251028 21:11:49.593142 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:49.593498 19633 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:37102
I20251028 21:11:49.593577 19633 raft_consensus.cc:606] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:49.593719 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:49.593777 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:49.961438 20627 raft_consensus.cc:993] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:49.961527 20627 raft_consensus.cc:1081] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:49.961687 19707 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41426
I20251028 21:11:49.961756 19707 raft_consensus.cc:493] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:49.961782 19707 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:49.962716 19707 raft_consensus.cc:515] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:49.962890 19707 leader_election.cc:290] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 5 election: Requested vote from peers 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2:36457), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:49.963009 19633 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
I20251028 21:11:49.963106 19633 raft_consensus.cc:3055] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 LEADER]: Stepping down as leader of term 4
I20251028 21:11:49.963183 19633 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:11:49.963092 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 5 candidate_status { last_received { term: 4 index: 32 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:49.963235 19633 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:49.963290 19978 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:49.963325 19633 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Advancing to term 5
I20251028 21:11:49.963769 19633 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 5.
I20251028 21:11:49.963941 19678 leader_election.cc:304] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:49.964056 19978 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 5.
I20251028 21:11:49.964062 20620 raft_consensus.cc:2804] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Leader election won for term 5
I20251028 21:11:49.964241 20620 raft_consensus.cc:697] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:49.964324 20620 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 32, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:49.964764 19450 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 4 to 5, leader changed from 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 5 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:50.392788 19633 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 5 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251028 21:11:50.393007 20620 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:50.394037 19978 raft_consensus.cc:1275] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 5 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 5 index: 33. (index mismatch)
I20251028 21:11:50.394227 20639 consensus_queue.cc:1048] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251028 21:11:51.594038 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:51.594467 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:51.594560 19707 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 LEADER]: Received request to transfer leadership to TS 95350159593b4057ae155343f06246d0
I20251028 21:11:51.594745 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:51.594822 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:51.854574 20650 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:51.854647 20650 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 LEADER]: Signalling peer 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:51.854813 19558 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:53816
I20251028 21:11:51.854887 19558 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:51.854903 19558 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 6 FOLLOWER]: Advancing to term 7
I20251028 21:11:51.855513 19558 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:51.855628 19558 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 7 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:51.855754 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 7 candidate_status { last_received { term: 6 index: 34 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:51.855808 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 6 FOLLOWER]: Advancing to term 7
I20251028 21:11:51.855782 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 7 candidate_status { last_received { term: 6 index: 34 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:51.855882 19707 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 LEADER]: Stepping down as leader of term 6
I20251028 21:11:51.855914 19707 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:51.855978 19707 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 6.34, Last appended by leader: 34, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:51.856060 19707 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 FOLLOWER]: Advancing to term 7
I20251028 21:11:51.856429 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 7.
I20251028 21:11:51.856590 19707 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 7.
I20251028 21:11:51.856607 19524 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, 9e69ab942ba042b6b83938c67405973c; no voters:
I20251028 21:11:51.856709 20572 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 7 FOLLOWER]: Leader election won for term 7
I20251028 21:11:51.856797 20572 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 7 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:51.856851 20572 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 6.34, Last appended by leader: 34, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:51.857285 19450 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 6 to 7, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 7 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:52.420624 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 7 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 6 index: 34. Preceding OpId from leader: term: 7 index: 35. (index mismatch)
I20251028 21:11:52.420833 20572 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251028 21:11:52.421723 19707 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 7 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 6 index: 34. Preceding OpId from leader: term: 7 index: 35. (index mismatch)
I20251028 21:11:52.421931 20572 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251028 21:11:53.595065 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:53.595429 19558 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:53.595503 19558 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 7 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:53.595707 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:53.595757 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:53.959818 20572 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:53.959887 20572 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 7 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:53.960083 19707 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41428
I20251028 21:11:53.960156 19707 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:53.960175 19707 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 7 FOLLOWER]: Advancing to term 8
I20251028 21:11:53.960747 19707 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:53.960850 19707 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 8 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:53.960996 19558 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 8 candidate_status { last_received { term: 7 index: 35 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:53.961011 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 8 candidate_status { last_received { term: 7 index: 35 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:53.961082 19558 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 7 LEADER]: Stepping down as leader of term 7
I20251028 21:11:53.961088 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 7 FOLLOWER]: Advancing to term 8
I20251028 21:11:53.961123 19558 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:53.961172 19558 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 7.35, Last appended by leader: 35, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:53.961258 19558 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 7 FOLLOWER]: Advancing to term 8
I20251028 21:11:53.961763 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 8.
I20251028 21:11:53.961848 19558 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 8.
I20251028 21:11:53.961912 19675 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e69ab942ba042b6b83938c67405973c, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:53.961984 20650 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 8 FOLLOWER]: Leader election won for term 8
I20251028 21:11:53.962076 20650 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 8 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:53.962131 20650 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 7.35, Last appended by leader: 35, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:53.962632 19452 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 7 to 8, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 8 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:54.436794 19558 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 8 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 7 index: 35. Preceding OpId from leader: term: 8 index: 36. (index mismatch)
I20251028 21:11:54.437009 20650 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251028 21:11:54.438102 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 8 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 7 index: 35. Preceding OpId from leader: term: 8 index: 36. (index mismatch)
I20251028 21:11:54.438460 20650 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251028 21:11:55.595902 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:55.596242 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:55.596309 19707 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 8 LEADER]: Received request to transfer leadership to TS 95350159593b4057ae155343f06246d0
I20251028 21:11:55.596473 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:55.596521 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:55.971436 20650 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:55.971534 20650 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 8 LEADER]: Signalling peer 95350159593b4057ae155343f06246d0 to start an election
I20251028 21:11:55.971719 19559 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
from {username='slave'} at 127.0.0.1:53816
I20251028 21:11:55.971796 19559 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:55.971818 19559 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 8 FOLLOWER]: Advancing to term 9
I20251028 21:11:55.972433 19559 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:55.972568 19559 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 9 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:55.972707 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 9 candidate_status { last_received { term: 8 index: 36 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:11:55.972767 19707 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 8 LEADER]: Stepping down as leader of term 8
I20251028 21:11:55.972731 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "95350159593b4057ae155343f06246d0" candidate_term: 9 candidate_status { last_received { term: 8 index: 36 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:55.972785 19707 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:55.972805 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 8 FOLLOWER]: Advancing to term 9
I20251028 21:11:55.972834 19707 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 8.36, Last appended by leader: 36, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:55.972920 19707 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 8 FOLLOWER]: Advancing to term 9
I20251028 21:11:55.973377 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 9.
I20251028 21:11:55.973423 19707 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95350159593b4057ae155343f06246d0 in term 9.
I20251028 21:11:55.973580 19524 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95350159593b4057ae155343f06246d0, 9e69ab942ba042b6b83938c67405973c; no voters:
I20251028 21:11:55.973681 20572 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 9 FOLLOWER]: Leader election won for term 9
I20251028 21:11:55.973788 20572 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 9 LEADER]: Becoming Leader. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:55.973837 20572 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 8.36, Last appended by leader: 36, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:55.974274 19452 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 reported cstate change: term changed from 8 to 9, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 95350159593b4057ae155343f06246d0 (127.15.205.1). New cstate: current_term: 9 leader_uuid: "95350159593b4057ae155343f06246d0" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:56.401293 19707 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 9 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 8 index: 36. Preceding OpId from leader: term: 9 index: 37. (index mismatch)
I20251028 21:11:56.401484 20572 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251028 21:11:56.402390 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 9 FOLLOWER]: Refusing update from remote peer 95350159593b4057ae155343f06246d0: Log matching property violated. Preceding OpId in replica: term: 8 index: 36. Preceding OpId from leader: term: 9 index: 37. (index mismatch)
I20251028 21:11:56.402709 20572 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251028 21:11:57.596660 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:57.597028 19559 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "95350159593b4057ae155343f06246d0"
mode: GRACEFUL
new_leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:53834
I20251028 21:11:57.597097 19559 raft_consensus.cc:606] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 9 LEADER]: Received request to transfer leadership to TS ac203379ff4e4fdbac249bcd15cf5be1
I20251028 21:11:57.597246 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:57.597311 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:11:57.816375 20690 raft_consensus.cc:993] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: : Instructing follower ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:57.816448 20690 raft_consensus.cc:1081] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 9 LEADER]: Signalling peer ac203379ff4e4fdbac249bcd15cf5be1 to start an election
I20251028 21:11:57.816618 19707 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
from {username='slave'} at 127.0.0.1:41428
I20251028 21:11:57.816679 19707 raft_consensus.cc:493] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:11:57.816699 19707 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 9 FOLLOWER]: Advancing to term 10
I20251028 21:11:57.817282 19707 raft_consensus.cc:515] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:57.817400 19707 leader_election.cc:290] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 10 election: Requested vote from peers 95350159593b4057ae155343f06246d0 (127.15.205.1:38429), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:11:57.817550 19559 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 10 candidate_status { last_received { term: 9 index: 37 } } ignore_live_leader: true dest_uuid: "95350159593b4057ae155343f06246d0"
I20251028 21:11:57.817564 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e83bd2a0bd3c40c1b6abb84a48fa9013" candidate_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" candidate_term: 10 candidate_status { last_received { term: 9 index: 37 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:11:57.817623 19559 raft_consensus.cc:3055] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 9 LEADER]: Stepping down as leader of term 9
I20251028 21:11:57.817638 19978 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 9 FOLLOWER]: Advancing to term 10
I20251028 21:11:57.817641 19559 raft_consensus.cc:740] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 95350159593b4057ae155343f06246d0, State: Running, Role: LEADER
I20251028 21:11:57.817710 19559 consensus_queue.cc:260] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 9.37, Last appended by leader: 37, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:57.817773 19559 raft_consensus.cc:3060] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 9 FOLLOWER]: Advancing to term 10
I20251028 21:11:57.818188 19978 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 10.
I20251028 21:11:57.818295 19559 raft_consensus.cc:2468] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac203379ff4e4fdbac249bcd15cf5be1 in term 10.
I20251028 21:11:57.818346 19675 leader_election.cc:304] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e69ab942ba042b6b83938c67405973c, ac203379ff4e4fdbac249bcd15cf5be1; no voters:
I20251028 21:11:57.818418 20650 raft_consensus.cc:2804] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 10 FOLLOWER]: Leader election won for term 10
I20251028 21:11:57.818518 20650 raft_consensus.cc:697] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 10 LEADER]: Becoming Leader. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:11:57.818573 20650 consensus_queue.cc:237] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 9.37, Last appended by leader: 37, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:11:57.819010 19450 catalog_manager.cc:5649] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 reported cstate change: term changed from 9 to 10, leader changed from 95350159593b4057ae155343f06246d0 (127.15.205.1) to ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3). New cstate: current_term: 10 leader_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:11:58.199748 19978 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 10 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 9 index: 37. Preceding OpId from leader: term: 10 index: 38. (index mismatch)
I20251028 21:11:58.199988 20650 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251028 21:11:58.201035 19559 raft_consensus.cc:1275] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 10 FOLLOWER]: Refusing update from remote peer ac203379ff4e4fdbac249bcd15cf5be1: Log matching property violated. Preceding OpId in replica: term: 9 index: 37. Preceding OpId from leader: term: 10 index: 38. (index mismatch)
I20251028 21:11:58.201224 20692 consensus_queue.cc:1048] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95350159593b4057ae155343f06246d0" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251028 21:11:59.597512 19512 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:11:59.597934 19707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
mode: GRACEFUL
new_leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:41470
I20251028 21:11:59.598009 19707 raft_consensus.cc:606] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 LEADER]: Received request to transfer leadership to TS 6b99aaed3fec4c28bf3e3dbbd676ab65
I20251028 21:11:59.598233 19512 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:11:59.598297 19512 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:12:00.061506 20650 raft_consensus.cc:993] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: : Instructing follower 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:12:00.061628 20650 raft_consensus.cc:1081] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 LEADER]: Signalling peer 6b99aaed3fec4c28bf3e3dbbd676ab65 to start an election
I20251028 21:12:00.061796 19633 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c"
dest_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65"
from {username='slave'} at 127.0.0.1:37080
I20251028 21:12:00.061877 19633 raft_consensus.cc:493] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:12:00.061897 19633 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 5 FOLLOWER]: Advancing to term 6
I20251028 21:12:00.062503 19633 raft_consensus.cc:515] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:12:00.062647 19633 leader_election.cc:290] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 6 election: Requested vote from peers ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3:42465), 9e69ab942ba042b6b83938c67405973c (127.15.205.4:42831)
I20251028 21:12:00.062808 19707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "ac203379ff4e4fdbac249bcd15cf5be1"
I20251028 21:12:00.062826 19978 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d738dd2ed0444dfeb4ff5cafec9c1d2c" candidate_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: true dest_uuid: "9e69ab942ba042b6b83938c67405973c"
I20251028 21:12:00.062889 19707 raft_consensus.cc:3055] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 LEADER]: Stepping down as leader of term 5
I20251028 21:12:00.062896 19978 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 5 FOLLOWER]: Advancing to term 6
I20251028 21:12:00.062947 19707 raft_consensus.cc:740] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ac203379ff4e4fdbac249bcd15cf5be1, State: Running, Role: LEADER
I20251028 21:12:00.063028 19707 consensus_queue.cc:260] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:12:00.063117 19707 raft_consensus.cc:3060] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Advancing to term 6
I20251028 21:12:00.063671 19978 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 6.
I20251028 21:12:00.063804 19707 raft_consensus.cc:2468] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b99aaed3fec4c28bf3e3dbbd676ab65 in term 6.
I20251028 21:12:00.063854 19600 leader_election.cc:304] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b99aaed3fec4c28bf3e3dbbd676ab65, 9e69ab942ba042b6b83938c67405973c; no voters:
I20251028 21:12:00.063966 20627 raft_consensus.cc:2804] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 6 FOLLOWER]: Leader election won for term 6
I20251028 21:12:00.064067 20627 raft_consensus.cc:697] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 6 LEADER]: Becoming Leader. State: Replica: 6b99aaed3fec4c28bf3e3dbbd676ab65, State: Running, Role: LEADER
I20251028 21:12:00.064122 20627 consensus_queue.cc:237] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } }
I20251028 21:12:00.064599 19450 catalog_manager.cc:5649] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 reported cstate change: term changed from 5 to 6, leader changed from ac203379ff4e4fdbac249bcd15cf5be1 (127.15.205.3) to 6b99aaed3fec4c28bf3e3dbbd676ab65 (127.15.205.2). New cstate: current_term: 6 leader_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "6b99aaed3fec4c28bf3e3dbbd676ab65" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac203379ff4e4fdbac249bcd15cf5be1" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e69ab942ba042b6b83938c67405973c" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42831 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/auto_rebalancer-test.cc:196: Failure
Expected equality of these values:
0
NumMovesScheduled(leader_idx, BalanceThreadType::LEADER_REBALANCE)
Which is: 1
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/auto_rebalancer-test.cc:575: Failure
Expected: CheckNoLeaderMovesScheduled() doesn't generate new fatal failures in the current thread.
Actual: it does.
I20251028 21:12:00.138960 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:12:00.142048 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:00.142167 16180 tablet_replica.cc:333] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:12:00.142218 16180 raft_consensus.cc:2243] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.142278 16180 raft_consensus.cc:2272] T 4c67b4b90e724c8492a4c82e84487f23 P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.142463 16180 tablet_replica.cc:333] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:12:00.142498 16180 raft_consensus.cc:2243] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.142530 16180 raft_consensus.cc:2272] T b22601bb9db44a3489691047e5a7c40a P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.142640 16180 tablet_replica.cc:333] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:12:00.142669 16180 raft_consensus.cc:2243] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 10 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.142704 16180 raft_consensus.cc:2272] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 95350159593b4057ae155343f06246d0 [term 10 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.142808 16180 tablet_replica.cc:333] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:12:00.142835 16180 raft_consensus.cc:2243] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.142896 16180 raft_consensus.cc:2272] T 10b6448e96bb4fd08006249ba36d650e P 95350159593b4057ae155343f06246d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.143033 16180 tablet_replica.cc:333] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:12:00.143069 16180 raft_consensus.cc:2243] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 5 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.143126 16180 raft_consensus.cc:2272] T 352c91b410644a06914dd52b1ba5c584 P 95350159593b4057ae155343f06246d0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.143239 16180 tablet_replica.cc:333] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0: stopping tablet replica
I20251028 21:12:00.143296 16180 raft_consensus.cc:2243] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.143335 16180 raft_consensus.cc:2272] T 903511324c64491b827a522851cb2402 P 95350159593b4057ae155343f06246d0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.155654 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:12:00.156924 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:12:00.160665 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:00.160777 16180 tablet_replica.cc:333] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:12:00.160835 16180 raft_consensus.cc:2243] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.160904 16180 raft_consensus.cc:2272] T b22601bb9db44a3489691047e5a7c40a P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.161034 16180 tablet_replica.cc:333] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:12:00.161072 16180 raft_consensus.cc:2243] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.161124 16180 raft_consensus.cc:2272] T ff26670b4bcc4167867fa54b97417365 P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.161233 16180 tablet_replica.cc:333] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:12:00.161265 16180 raft_consensus.cc:2243] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.161314 16180 raft_consensus.cc:2272] T 10b6448e96bb4fd08006249ba36d650e P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.161417 16180 tablet_replica.cc:333] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:12:00.161449 16180 raft_consensus.cc:2243] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.161499 16180 raft_consensus.cc:2272] T 590f65c9d19a4688b25734528ac7474f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.161607 16180 tablet_replica.cc:333] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:12:00.161639 16180 raft_consensus.cc:2243] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 6 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.161682 16180 pending_rounds.cc:70] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: Trying to abort 1 pending ops.
I20251028 21:12:00.161720 16180 pending_rounds.cc:77] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65: Aborting op as it isn't in flight: id { term: 6 index: 34 } timestamp: 7215865528583200768 op_type: NO_OP noop_request { }
I20251028 21:12:00.161787 16180 raft_consensus.cc:2889] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 6 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:00.161819 16180 raft_consensus.cc:2272] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.161983 16180 tablet_replica.cc:333] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65: stopping tablet replica
I20251028 21:12:00.162020 16180 raft_consensus.cc:2243] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.162072 16180 raft_consensus.cc:2272] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 6b99aaed3fec4c28bf3e3dbbd676ab65 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.174642 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:12:00.175933 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:12:00.179493 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:00.179581 16180 tablet_replica.cc:333] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:12:00.179638 16180 raft_consensus.cc:2243] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.179687 16180 raft_consensus.cc:2272] T f49b5a0ce6fe4d91aa0ab60a1d770949 P ac203379ff4e4fdbac249bcd15cf5be1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.179792 16180 tablet_replica.cc:333] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:12:00.179821 16180 raft_consensus.cc:2243] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.179853 16180 raft_consensus.cc:2272] T 590f65c9d19a4688b25734528ac7474f P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.179931 16180 tablet_replica.cc:333] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:12:00.179955 16180 raft_consensus.cc:2243] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.179996 16180 raft_consensus.cc:2272] T 903511324c64491b827a522851cb2402 P ac203379ff4e4fdbac249bcd15cf5be1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.180066 16180 tablet_replica.cc:333] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:12:00.180088 16180 raft_consensus.cc:2243] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.180114 16180 raft_consensus.cc:2272] T 352c91b410644a06914dd52b1ba5c584 P ac203379ff4e4fdbac249bcd15cf5be1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.180192 16180 tablet_replica.cc:333] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:12:00.180217 16180 raft_consensus.cc:2243] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 10 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.180249 16180 raft_consensus.cc:2272] T e83bd2a0bd3c40c1b6abb84a48fa9013 P ac203379ff4e4fdbac249bcd15cf5be1 [term 10 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.180322 16180 tablet_replica.cc:333] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1: stopping tablet replica
I20251028 21:12:00.180346 16180 raft_consensus.cc:2243] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.180367 16180 raft_consensus.cc:2272] T d738dd2ed0444dfeb4ff5cafec9c1d2c P ac203379ff4e4fdbac249bcd15cf5be1 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.192036 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:12:00.193298 16180 tablet_server.cc:178] TabletServer@127.15.205.4:0 shutting down...
I20251028 21:12:00.196238 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:00.196329 16180 tablet_replica.cc:333] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c: stopping tablet replica
I20251028 21:12:00.196367 16180 raft_consensus.cc:2243] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.196408 16180 raft_consensus.cc:2272] T e684839cc93d4e4685039496fbd01e03 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.196506 16180 tablet_replica.cc:333] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c: stopping tablet replica
I20251028 21:12:00.196533 16180 raft_consensus.cc:2243] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.196566 16180 raft_consensus.cc:2272] T 10b6448e96bb4fd08006249ba36d650e P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.196641 16180 tablet_replica.cc:333] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c: stopping tablet replica
I20251028 21:12:00.196666 16180 raft_consensus.cc:2243] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 4 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.196699 16180 raft_consensus.cc:2272] T 903511324c64491b827a522851cb2402 P 9e69ab942ba042b6b83938c67405973c [term 4 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.196776 16180 tablet_replica.cc:333] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c: stopping tablet replica
I20251028 21:12:00.196797 16180 raft_consensus.cc:2243] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 6 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.196826 16180 raft_consensus.cc:2272] T d738dd2ed0444dfeb4ff5cafec9c1d2c P 9e69ab942ba042b6b83938c67405973c [term 6 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.196900 16180 tablet_replica.cc:333] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c: stopping tablet replica
I20251028 21:12:00.196923 16180 raft_consensus.cc:2243] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 10 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.196947 16180 raft_consensus.cc:2272] T e83bd2a0bd3c40c1b6abb84a48fa9013 P 9e69ab942ba042b6b83938c67405973c [term 10 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.197047 16180 tablet_replica.cc:333] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c: stopping tablet replica
I20251028 21:12:00.197072 16180 raft_consensus.cc:2243] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.197103 16180 raft_consensus.cc:2272] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 9e69ab942ba042b6b83938c67405973c [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.208340 16180 tablet_server.cc:195] TabletServer@127.15.205.4:0 shutdown complete.
I20251028 21:12:00.209317 16180 tablet_server.cc:178] TabletServer@127.15.205.5:0 shutting down...
I20251028 21:12:00.212320 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:00.212419 16180 tablet_replica.cc:333] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0: stopping tablet replica
I20251028 21:12:00.212469 16180 raft_consensus.cc:2243] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.212519 16180 raft_consensus.cc:2272] T f49b5a0ce6fe4d91aa0ab60a1d770949 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.212642 16180 tablet_replica.cc:333] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0: stopping tablet replica
I20251028 21:12:00.212678 16180 raft_consensus.cc:2243] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.212725 16180 raft_consensus.cc:2272] T 4c67b4b90e724c8492a4c82e84487f23 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.212827 16180 tablet_replica.cc:333] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0: stopping tablet replica
I20251028 21:12:00.212862 16180 raft_consensus.cc:2243] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 5 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.212905 16180 raft_consensus.cc:2272] T 352c91b410644a06914dd52b1ba5c584 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.213029 16180 tablet_replica.cc:333] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0: stopping tablet replica
I20251028 21:12:00.213063 16180 raft_consensus.cc:2243] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.213119 16180 raft_consensus.cc:2272] T ff26670b4bcc4167867fa54b97417365 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.213229 16180 tablet_replica.cc:333] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0: stopping tablet replica
I20251028 21:12:00.213263 16180 raft_consensus.cc:2243] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.213325 16180 raft_consensus.cc:2272] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.213431 16180 tablet_replica.cc:333] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0: stopping tablet replica
I20251028 21:12:00.213462 16180 raft_consensus.cc:2243] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.213532 16180 raft_consensus.cc:2272] T e684839cc93d4e4685039496fbd01e03 P 05d39c90d87d4cca9a23f40ad31b12e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.225591 16180 tablet_server.cc:195] TabletServer@127.15.205.5:0 shutdown complete.
I20251028 21:12:00.226524 16180 tablet_server.cc:178] TabletServer@127.15.205.6:0 shutting down...
I20251028 21:12:00.230659 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:00.230747 16180 tablet_replica.cc:333] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf: stopping tablet replica
I20251028 21:12:00.230810 16180 raft_consensus.cc:2243] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.230856 16180 raft_consensus.cc:2272] T ff26670b4bcc4167867fa54b97417365 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.230964 16180 tablet_replica.cc:333] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf: stopping tablet replica
I20251028 21:12:00.230995 16180 raft_consensus.cc:2243] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.231027 16180 raft_consensus.cc:2272] T 8798d7cdf7864b0bbd9e5b2e26d0dc4f P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.231105 16180 tablet_replica.cc:333] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf: stopping tablet replica
I20251028 21:12:00.231132 16180 raft_consensus.cc:2243] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.231181 16180 raft_consensus.cc:2272] T e684839cc93d4e4685039496fbd01e03 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.231258 16180 tablet_replica.cc:333] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf: stopping tablet replica
I20251028 21:12:00.231350 16180 raft_consensus.cc:2243] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.231395 16180 raft_consensus.cc:2272] T 4c67b4b90e724c8492a4c82e84487f23 P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.231479 16180 tablet_replica.cc:333] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf: stopping tablet replica
I20251028 21:12:00.231505 16180 raft_consensus.cc:2243] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 4 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.231537 16180 raft_consensus.cc:2272] T 590f65c9d19a4688b25734528ac7474f P a39fb1f187c544f2b77164040db04caf [term 4 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.231611 16180 tablet_replica.cc:333] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf: stopping tablet replica
I20251028 21:12:00.231634 16180 raft_consensus.cc:2243] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:00.231664 16180 raft_consensus.cc:2272] T b22601bb9db44a3489691047e5a7c40a P a39fb1f187c544f2b77164040db04caf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.243383 16180 tablet_server.cc:195] TabletServer@127.15.205.6:0 shutdown complete.
I20251028 21:12:00.244334 16180 master.cc:561] Master@127.15.205.62:41741 shutting down...
I20251028 21:12:00.246623 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:00.246673 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:00.246690 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 47d74768e36048b29dc382adb5d0bf36: stopping tablet replica
I20251028 21:12:00.258860 16180 master.cc:583] Master@127.15.205.62:41741 shutdown complete.
I20251028 21:12:00.260463 16180 test_util.cc:183] -----------------------------------------------
I20251028 21:12:00.260506 16180 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1761685792944951-16180-0
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer (34733 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingUnstableCluster
I20251028 21:12:00.260869 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:46785
I20251028 21:12:00.260982 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:12:00.262221 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:00.262285 20712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:00.262332 20711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:00.262466 20714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:00.262586 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:00.262614 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:00.262632 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685920262632 us; error 0 us; skew 500 ppm
I20251028 21:12:00.263159 16180 webserver.cc:492] Webserver started at http://127.15.205.62:42165/ using document root <none> and password file <none>
I20251028 21:12:00.263255 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:00.263329 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:00.263374 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:00.263602 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "e09b9fa3084d4b459268d2f0f031480a"
format_stamp: "Formatted at 2025-10-28 21:12:00 on dist-test-slave-zmkx"
I20251028 21:12:00.264385 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:00.264729 20719 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.264835 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.264884 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "e09b9fa3084d4b459268d2f0f031480a"
format_stamp: "Formatted at 2025-10-28 21:12:00 on dist-test-slave-zmkx"
I20251028 21:12:00.264912 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:00.271566 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:00.271685 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:00.273881 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:46785
I20251028 21:12:00.280118 20771 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:46785 every 8 connection(s)
I20251028 21:12:00.280130 20772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:00.280941 20772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a: Bootstrap starting.
I20251028 21:12:00.281157 20772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:00.281567 20772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a: No bootstrap required, opened a new log
I20251028 21:12:00.281649 20772 raft_consensus.cc:359] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09b9fa3084d4b459268d2f0f031480a" member_type: VOTER }
I20251028 21:12:00.281684 20772 raft_consensus.cc:385] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:00.281697 20772 raft_consensus.cc:740] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e09b9fa3084d4b459268d2f0f031480a, State: Initialized, Role: FOLLOWER
I20251028 21:12:00.281726 20772 consensus_queue.cc:260] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09b9fa3084d4b459268d2f0f031480a" member_type: VOTER }
I20251028 21:12:00.281766 20772 raft_consensus.cc:399] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:12:00.281782 20772 raft_consensus.cc:493] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:12:00.281798 20772 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:00.282196 20772 raft_consensus.cc:515] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09b9fa3084d4b459268d2f0f031480a" member_type: VOTER }
I20251028 21:12:00.282241 20772 leader_election.cc:304] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e09b9fa3084d4b459268d2f0f031480a; no voters:
I20251028 21:12:00.282312 20772 leader_election.cc:290] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:12:00.282359 20775 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:00.282406 20772 sys_catalog.cc:565] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:12:00.282483 20775 raft_consensus.cc:697] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 1 LEADER]: Becoming Leader. State: Replica: e09b9fa3084d4b459268d2f0f031480a, State: Running, Role: LEADER
I20251028 21:12:00.282541 20775 consensus_queue.cc:237] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09b9fa3084d4b459268d2f0f031480a" member_type: VOTER }
I20251028 21:12:00.282712 20776 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e09b9fa3084d4b459268d2f0f031480a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09b9fa3084d4b459268d2f0f031480a" member_type: VOTER } }
I20251028 21:12:00.282740 20777 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e09b9fa3084d4b459268d2f0f031480a. Latest consensus state: current_term: 1 leader_uuid: "e09b9fa3084d4b459268d2f0f031480a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09b9fa3084d4b459268d2f0f031480a" member_type: VOTER } }
I20251028 21:12:00.282833 20776 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:00.282845 20777 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:00.283032 20783 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:12:00.283164 20783 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:12:00.283689 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:12:00.283694 20783 catalog_manager.cc:1357] Generated new cluster ID: dd691b808b1443388b11b62ef22cf570
I20251028 21:12:00.283754 20783 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:12:00.294478 20783 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:12:00.294766 20783 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:12:00.301712 20783 catalog_manager.cc:6022] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a: Generated new TSK 0
I20251028 21:12:00.301761 20783 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:12:00.315433 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:00.316449 20793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:00.316561 20794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:00.316612 20796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:00.316685 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:00.316773 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:00.316797 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:00.316818 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685920316818 us; error 0 us; skew 500 ppm
I20251028 21:12:00.317404 16180 webserver.cc:492] Webserver started at http://127.15.205.1:42749/ using document root <none> and password file <none>
I20251028 21:12:00.317474 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:00.317515 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:00.317554 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:00.317750 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "962bf735c56f43029889df54294c897b"
format_stamp: "Formatted at 2025-10-28 21:12:00 on dist-test-slave-zmkx"
I20251028 21:12:00.318482 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:00.318830 20801 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.318946 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.318987 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "962bf735c56f43029889df54294c897b"
format_stamp: "Formatted at 2025-10-28 21:12:00 on dist-test-slave-zmkx"
I20251028 21:12:00.319024 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:00.325232 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:00.325347 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:00.325443 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:00.325560 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:00.325582 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.325604 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:00.325620 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.328202 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:40907
I20251028 21:12:00.328713 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:12:00.328748 20864 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:40907 every 8 connection(s)
W20251028 21:12:00.329712 20870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:00.329842 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:00.329727 20868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:00.329818 20872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:00.330051 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:00.330089 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:00.330122 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685920330123 us; error 0 us; skew 500 ppm
I20251028 21:12:00.330585 16180 webserver.cc:492] Webserver started at http://127.15.205.2:42271/ using document root <none> and password file <none>
I20251028 21:12:00.330654 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:00.330693 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:00.330734 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:00.330986 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "8600621808894b91a0654602a14118c8"
format_stamp: "Formatted at 2025-10-28 21:12:00 on dist-test-slave-zmkx"
I20251028 21:12:00.332005 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:00.332368 20877 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.332485 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.332516 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "8600621808894b91a0654602a14118c8"
format_stamp: "Formatted at 2025-10-28 21:12:00 on dist-test-slave-zmkx"
I20251028 21:12:00.332540 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:00.336243 20865 heartbeater.cc:344] Connected to a master server at 127.15.205.62:46785
I20251028 21:12:00.336293 20865 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:00.336378 20865 heartbeater.cc:507] Master 127.15.205.62:46785 requested a full tablet report, sending...
I20251028 21:12:00.336557 20736 ts_manager.cc:194] Registered new tserver with Master: 962bf735c56f43029889df54294c897b (127.15.205.1:40907)
I20251028 21:12:00.337016 20736 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52228
I20251028 21:12:00.338946 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:00.339092 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:00.339193 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:00.339347 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:00.339370 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.339386 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:00.339411 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.341938 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:36673
I20251028 21:12:00.341964 20940 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:36673 every 8 connection(s)
I20251028 21:12:00.342238 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:00.343142 20944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:00.343215 20945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:00.343483 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:00.343633 20947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:00.343811 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:00.343844 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:00.343866 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685920343866 us; error 0 us; skew 500 ppm
I20251028 21:12:00.344383 16180 webserver.cc:492] Webserver started at http://127.15.205.3:42087/ using document root <none> and password file <none>
I20251028 21:12:00.344470 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:00.344545 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:00.344597 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:00.344836 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d"
format_stamp: "Formatted at 2025-10-28 21:12:00 on dist-test-slave-zmkx"
I20251028 21:12:00.345214 20941 heartbeater.cc:344] Connected to a master server at 127.15.205.62:46785
I20251028 21:12:00.345271 20941 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:00.345350 20941 heartbeater.cc:507] Master 127.15.205.62:46785 requested a full tablet report, sending...
I20251028 21:12:00.345561 20736 ts_manager.cc:194] Registered new tserver with Master: 8600621808894b91a0654602a14118c8 (127.15.205.2:36673)
I20251028 21:12:00.345770 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:00.345863 20736 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52238
I20251028 21:12:00.346252 20952 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.346366 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.346414 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d"
format_stamp: "Formatted at 2025-10-28 21:12:00 on dist-test-slave-zmkx"
I20251028 21:12:00.346458 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:00.353627 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:00.353796 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:00.353921 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:00.354115 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:00.354153 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.354180 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:00.354207 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:00.356765 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:42901
I20251028 21:12:00.356873 21015 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:42901 every 8 connection(s)
I20251028 21:12:00.359884 21016 heartbeater.cc:344] Connected to a master server at 127.15.205.62:46785
I20251028 21:12:00.359936 21016 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:00.360025 21016 heartbeater.cc:507] Master 127.15.205.62:46785 requested a full tablet report, sending...
I20251028 21:12:00.360219 20736 ts_manager.cc:194] Registered new tserver with Master: 4ec21c1b6e8f4f6f9c4000b92a9bd62d (127.15.205.3:42901)
I20251028 21:12:00.360280 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00333754s
I20251028 21:12:00.360513 20736 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52240
I20251028 21:12:01.337515 20865 heartbeater.cc:499] Master 127.15.205.62:46785 was elected leader, sending a full tablet report...
I20251028 21:12:01.346308 20941 heartbeater.cc:499] Master 127.15.205.62:46785 was elected leader, sending a full tablet report...
I20251028 21:12:01.360936 21016 heartbeater.cc:499] Master 127.15.205.62:46785 was elected leader, sending a full tablet report...
I20251028 21:12:01.385228 16180 test_util.cc:276] Using random seed: 2005715723
I20251028 21:12:01.389860 20736 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52256:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:12:01.389999 20736 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:12:01.394881 20905 tablet_service.cc:1505] Processing CreateTablet for tablet d573c542ddc24556a8142e2b2a56f12a (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251028 21:12:01.394902 20904 tablet_service.cc:1505] Processing CreateTablet for tablet 9a4169e5554c4b9c951b8a947ad974c2 (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251028 21:12:01.395020 20905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d573c542ddc24556a8142e2b2a56f12a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.395172 20904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a4169e5554c4b9c951b8a947ad974c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.396425 21036 tablet_bootstrap.cc:492] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8: Bootstrap starting.
I20251028 21:12:01.396430 20903 tablet_service.cc:1505] Processing CreateTablet for tablet 2b489548a0414e52b8d8a0af293e9ab1 (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251028 21:12:01.396620 20903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b489548a0414e52b8d8a0af293e9ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.396654 20979 tablet_service.cc:1505] Processing CreateTablet for tablet 9a4169e5554c4b9c951b8a947ad974c2 (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251028 21:12:01.396736 20979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a4169e5554c4b9c951b8a947ad974c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.396950 21036 tablet_bootstrap.cc:654] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.397450 21036 tablet_bootstrap.cc:492] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8: No bootstrap required, opened a new log
I20251028 21:12:01.397490 21036 ts_tablet_manager.cc:1403] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:01.397636 21036 raft_consensus.cc:359] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.397694 21036 raft_consensus.cc:385] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.397712 21036 raft_consensus.cc:740] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600621808894b91a0654602a14118c8, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.397753 21036 consensus_queue.cc:260] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.396626 20980 tablet_service.cc:1505] Processing CreateTablet for tablet d573c542ddc24556a8142e2b2a56f12a (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251028 21:12:01.397902 21036 ts_tablet_manager.cc:1434] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.397902 20980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d573c542ddc24556a8142e2b2a56f12a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.397977 21036 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: Bootstrap starting.
I20251028 21:12:01.398283 20978 tablet_service.cc:1505] Processing CreateTablet for tablet 2b489548a0414e52b8d8a0af293e9ab1 (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251028 21:12:01.398348 21036 tablet_bootstrap.cc:654] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.398370 20978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b489548a0414e52b8d8a0af293e9ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.398808 20829 tablet_service.cc:1505] Processing CreateTablet for tablet d573c542ddc24556a8142e2b2a56f12a (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251028 21:12:01.398912 20829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d573c542ddc24556a8142e2b2a56f12a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.399940 21041 tablet_bootstrap.cc:492] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b: Bootstrap starting.
I20251028 21:12:01.399971 20827 tablet_service.cc:1505] Processing CreateTablet for tablet 2b489548a0414e52b8d8a0af293e9ab1 (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251028 21:12:01.399945 20828 tablet_service.cc:1505] Processing CreateTablet for tablet 9a4169e5554c4b9c951b8a947ad974c2 (DEFAULT_TABLE table=test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251028 21:12:01.400110 20827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b489548a0414e52b8d8a0af293e9ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.400245 20828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a4169e5554c4b9c951b8a947ad974c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:01.400287 21036 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: No bootstrap required, opened a new log
I20251028 21:12:01.400317 21036 ts_tablet_manager.cc:1403] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:01.400406 21041 tablet_bootstrap.cc:654] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.400452 21036 raft_consensus.cc:359] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.400498 21036 raft_consensus.cc:385] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.400514 21036 raft_consensus.cc:740] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600621808894b91a0654602a14118c8, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.400550 21036 consensus_queue.cc:260] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.400666 21036 ts_tablet_manager.cc:1434] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.400712 21036 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8: Bootstrap starting.
I20251028 21:12:01.401057 21036 tablet_bootstrap.cc:654] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.401623 21036 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8: No bootstrap required, opened a new log
I20251028 21:12:01.401659 21036 ts_tablet_manager.cc:1403] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:01.401767 21036 raft_consensus.cc:359] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.401822 21036 raft_consensus.cc:385] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.401847 21036 raft_consensus.cc:740] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600621808894b91a0654602a14118c8, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.401898 21036 consensus_queue.cc:260] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.402035 21036 ts_tablet_manager.cc:1434] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.402182 21039 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Bootstrap starting.
I20251028 21:12:01.402575 21039 tablet_bootstrap.cc:654] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.403050 21039 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: No bootstrap required, opened a new log
I20251028 21:12:01.403056 21041 tablet_bootstrap.cc:492] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b: No bootstrap required, opened a new log
I20251028 21:12:01.403136 21039 ts_tablet_manager.cc:1403] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:01.403162 21041 ts_tablet_manager.cc:1403] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:12:01.403309 21039 raft_consensus.cc:359] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.403362 21039 raft_consensus.cc:385] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.403349 21041 raft_consensus.cc:359] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.403386 21039 raft_consensus.cc:740] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec21c1b6e8f4f6f9c4000b92a9bd62d, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.403420 21041 raft_consensus.cc:385] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.403450 21041 raft_consensus.cc:740] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962bf735c56f43029889df54294c897b, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.403466 21039 consensus_queue.cc:260] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.403502 21041 consensus_queue.cc:260] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.403605 21039 ts_tablet_manager.cc:1434] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:01.403663 21041 ts_tablet_manager.cc:1434] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:01.403715 21041 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: Bootstrap starting.
I20251028 21:12:01.403739 21039 tablet_bootstrap.cc:492] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Bootstrap starting.
I20251028 21:12:01.404130 21041 tablet_bootstrap.cc:654] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.404146 21039 tablet_bootstrap.cc:654] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.404731 21041 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: No bootstrap required, opened a new log
I20251028 21:12:01.404769 21041 ts_tablet_manager.cc:1403] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:01.404834 21039 tablet_bootstrap.cc:492] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: No bootstrap required, opened a new log
I20251028 21:12:01.404889 21039 ts_tablet_manager.cc:1403] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:01.404883 21041 raft_consensus.cc:359] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.404968 21041 raft_consensus.cc:385] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.404997 21041 raft_consensus.cc:740] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962bf735c56f43029889df54294c897b, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.405045 21041 consensus_queue.cc:260] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.405061 21039 raft_consensus.cc:359] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.405118 21039 raft_consensus.cc:385] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.405123 21041 ts_tablet_manager.cc:1434] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.405151 21039 raft_consensus.cc:740] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec21c1b6e8f4f6f9c4000b92a9bd62d, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.405212 21041 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b: Bootstrap starting.
I20251028 21:12:01.405242 21039 consensus_queue.cc:260] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.405329 21039 ts_tablet_manager.cc:1434] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.405447 21039 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Bootstrap starting.
I20251028 21:12:01.405628 21041 tablet_bootstrap.cc:654] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.405854 21039 tablet_bootstrap.cc:654] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:01.406210 21041 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b: No bootstrap required, opened a new log
I20251028 21:12:01.406253 21041 ts_tablet_manager.cc:1403] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:01.406395 21041 raft_consensus.cc:359] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.406435 21039 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: No bootstrap required, opened a new log
I20251028 21:12:01.406476 21041 raft_consensus.cc:385] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.406522 21039 ts_tablet_manager.cc:1403] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:01.406529 21041 raft_consensus.cc:740] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962bf735c56f43029889df54294c897b, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.406584 21041 consensus_queue.cc:260] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.406665 21041 ts_tablet_manager.cc:1434] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.406670 21039 raft_consensus.cc:359] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.406754 21039 raft_consensus.cc:385] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:01.406805 21039 raft_consensus.cc:740] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec21c1b6e8f4f6f9c4000b92a9bd62d, State: Initialized, Role: FOLLOWER
I20251028 21:12:01.406852 21039 consensus_queue.cc:260] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.406944 21039 ts_tablet_manager.cc:1434] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.409809 21038 raft_consensus.cc:493] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:01.409854 21038 raft_consensus.cc:515] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.410009 21038 leader_election.cc:290] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 962bf735c56f43029889df54294c897b (127.15.205.1:40907), 4ec21c1b6e8f4f6f9c4000b92a9bd62d (127.15.205.3:42901)
I20251028 21:12:01.412375 20839 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b489548a0414e52b8d8a0af293e9ab1" candidate_uuid: "8600621808894b91a0654602a14118c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962bf735c56f43029889df54294c897b" is_pre_election: true
I20251028 21:12:01.412457 20839 raft_consensus.cc:2468] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600621808894b91a0654602a14118c8 in term 0.
I20251028 21:12:01.412565 20990 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b489548a0414e52b8d8a0af293e9ab1" candidate_uuid: "8600621808894b91a0654602a14118c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" is_pre_election: true
I20251028 21:12:01.412596 20878 leader_election.cc:304] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600621808894b91a0654602a14118c8, 962bf735c56f43029889df54294c897b; no voters:
I20251028 21:12:01.412613 20990 raft_consensus.cc:2468] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600621808894b91a0654602a14118c8 in term 0.
I20251028 21:12:01.412707 21038 raft_consensus.cc:2804] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:01.412740 21038 raft_consensus.cc:493] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:01.412770 21038 raft_consensus.cc:3060] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.413285 21038 raft_consensus.cc:515] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.413403 21038 leader_election.cc:290] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [CANDIDATE]: Term 1 election: Requested vote from peers 962bf735c56f43029889df54294c897b (127.15.205.1:40907), 4ec21c1b6e8f4f6f9c4000b92a9bd62d (127.15.205.3:42901)
I20251028 21:12:01.413520 20839 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b489548a0414e52b8d8a0af293e9ab1" candidate_uuid: "8600621808894b91a0654602a14118c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962bf735c56f43029889df54294c897b"
I20251028 21:12:01.413529 20990 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b489548a0414e52b8d8a0af293e9ab1" candidate_uuid: "8600621808894b91a0654602a14118c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d"
I20251028 21:12:01.413604 20839 raft_consensus.cc:3060] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.413615 20990 raft_consensus.cc:3060] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.414260 20839 raft_consensus.cc:2468] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600621808894b91a0654602a14118c8 in term 1.
I20251028 21:12:01.414269 20990 raft_consensus.cc:2468] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600621808894b91a0654602a14118c8 in term 1.
I20251028 21:12:01.414417 20880 leader_election.cc:304] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec21c1b6e8f4f6f9c4000b92a9bd62d, 8600621808894b91a0654602a14118c8; no voters:
I20251028 21:12:01.414511 21038 raft_consensus.cc:2804] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:01.414552 21038 raft_consensus.cc:697] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 1 LEADER]: Becoming Leader. State: Replica: 8600621808894b91a0654602a14118c8, State: Running, Role: LEADER
I20251028 21:12:01.414598 21038 consensus_queue.cc:237] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.415077 20736 catalog_manager.cc:5649] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8600621808894b91a0654602a14118c8 (127.15.205.2). New cstate: current_term: 1 leader_uuid: "8600621808894b91a0654602a14118c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:01.415642 21048 raft_consensus.cc:493] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:01.415697 21048 raft_consensus.cc:515] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.415830 21048 leader_election.cc:290] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ec21c1b6e8f4f6f9c4000b92a9bd62d (127.15.205.3:42901), 8600621808894b91a0654602a14118c8 (127.15.205.2:36673)
I20251028 21:12:01.418618 20990 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d573c542ddc24556a8142e2b2a56f12a" candidate_uuid: "962bf735c56f43029889df54294c897b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" is_pre_election: true
I20251028 21:12:01.418632 20915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d573c542ddc24556a8142e2b2a56f12a" candidate_uuid: "962bf735c56f43029889df54294c897b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600621808894b91a0654602a14118c8" is_pre_election: true
I20251028 21:12:01.418704 20990 raft_consensus.cc:2468] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962bf735c56f43029889df54294c897b in term 0.
I20251028 21:12:01.418708 20915 raft_consensus.cc:2468] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962bf735c56f43029889df54294c897b in term 0.
I20251028 21:12:01.418841 20804 leader_election.cc:304] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec21c1b6e8f4f6f9c4000b92a9bd62d, 962bf735c56f43029889df54294c897b; no voters:
I20251028 21:12:01.418912 21048 raft_consensus.cc:2804] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:01.418955 21048 raft_consensus.cc:493] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:01.418972 21048 raft_consensus.cc:3060] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.419441 21048 raft_consensus.cc:515] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.419535 21048 leader_election.cc:290] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [CANDIDATE]: Term 1 election: Requested vote from peers 4ec21c1b6e8f4f6f9c4000b92a9bd62d (127.15.205.3:42901), 8600621808894b91a0654602a14118c8 (127.15.205.2:36673)
I20251028 21:12:01.419700 20990 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d573c542ddc24556a8142e2b2a56f12a" candidate_uuid: "962bf735c56f43029889df54294c897b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d"
I20251028 21:12:01.419708 20915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d573c542ddc24556a8142e2b2a56f12a" candidate_uuid: "962bf735c56f43029889df54294c897b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600621808894b91a0654602a14118c8"
I20251028 21:12:01.419775 20990 raft_consensus.cc:3060] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.419780 20915 raft_consensus.cc:3060] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.420305 20990 raft_consensus.cc:2468] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962bf735c56f43029889df54294c897b in term 1.
I20251028 21:12:01.420305 20915 raft_consensus.cc:2468] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962bf735c56f43029889df54294c897b in term 1.
I20251028 21:12:01.420461 20805 leader_election.cc:304] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600621808894b91a0654602a14118c8, 962bf735c56f43029889df54294c897b; no voters:
I20251028 21:12:01.420547 21048 raft_consensus.cc:2804] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:01.420665 21048 raft_consensus.cc:697] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Becoming Leader. State: Replica: 962bf735c56f43029889df54294c897b, State: Running, Role: LEADER
I20251028 21:12:01.420707 21048 consensus_queue.cc:237] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:01.421147 20736 catalog_manager.cc:5649] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b reported cstate change: term changed from 0 to 1, leader changed from <none> to 962bf735c56f43029889df54294c897b (127.15.205.1). New cstate: current_term: 1 leader_uuid: "962bf735c56f43029889df54294c897b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:01.450007 21048 raft_consensus.cc:493] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:01.450076 21048 raft_consensus.cc:515] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.450104 21047 raft_consensus.cc:493] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:01.450184 21048 leader_election.cc:290] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8600621808894b91a0654602a14118c8 (127.15.205.2:36673), 4ec21c1b6e8f4f6f9c4000b92a9bd62d (127.15.205.3:42901)
I20251028 21:12:01.450158 21047 raft_consensus.cc:515] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.450299 20990 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a4169e5554c4b9c951b8a947ad974c2" candidate_uuid: "962bf735c56f43029889df54294c897b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" is_pre_election: true
I20251028 21:12:01.450305 20915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a4169e5554c4b9c951b8a947ad974c2" candidate_uuid: "962bf735c56f43029889df54294c897b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600621808894b91a0654602a14118c8" is_pre_election: true
I20251028 21:12:01.450335 21047 leader_election.cc:290] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 962bf735c56f43029889df54294c897b (127.15.205.1:40907), 8600621808894b91a0654602a14118c8 (127.15.205.2:36673)
I20251028 21:12:01.450397 20915 raft_consensus.cc:2468] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962bf735c56f43029889df54294c897b in term 0.
I20251028 21:12:01.450508 20805 leader_election.cc:304] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600621808894b91a0654602a14118c8, 962bf735c56f43029889df54294c897b; no voters:
I20251028 21:12:01.450368 20990 raft_consensus.cc:2468] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962bf735c56f43029889df54294c897b in term 0.
I20251028 21:12:01.450613 21048 raft_consensus.cc:2804] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:01.450635 21048 raft_consensus.cc:493] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:01.450662 21048 raft_consensus.cc:3060] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.451239 21048 raft_consensus.cc:515] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.451363 21048 leader_election.cc:290] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [CANDIDATE]: Term 1 election: Requested vote from peers 8600621808894b91a0654602a14118c8 (127.15.205.2:36673), 4ec21c1b6e8f4f6f9c4000b92a9bd62d (127.15.205.3:42901)
I20251028 21:12:01.451462 20915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a4169e5554c4b9c951b8a947ad974c2" candidate_uuid: "962bf735c56f43029889df54294c897b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600621808894b91a0654602a14118c8"
I20251028 21:12:01.451486 20990 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a4169e5554c4b9c951b8a947ad974c2" candidate_uuid: "962bf735c56f43029889df54294c897b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d"
I20251028 21:12:01.451525 20990 raft_consensus.cc:3060] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.451510 20915 raft_consensus.cc:3060] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:01.452096 20990 raft_consensus.cc:2468] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962bf735c56f43029889df54294c897b in term 1.
I20251028 21:12:01.452214 20915 raft_consensus.cc:2468] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962bf735c56f43029889df54294c897b in term 1.
I20251028 21:12:01.452221 20804 leader_election.cc:304] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec21c1b6e8f4f6f9c4000b92a9bd62d, 962bf735c56f43029889df54294c897b; no voters:
I20251028 21:12:01.452353 21048 raft_consensus.cc:2804] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:01.452399 21048 raft_consensus.cc:697] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Becoming Leader. State: Replica: 962bf735c56f43029889df54294c897b, State: Running, Role: LEADER
I20251028 21:12:01.452435 21048 consensus_queue.cc:237] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } }
I20251028 21:12:01.452805 20736 catalog_manager.cc:5649] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b reported cstate change: term changed from 0 to 1, leader changed from <none> to 962bf735c56f43029889df54294c897b (127.15.205.1). New cstate: current_term: 1 leader_uuid: "962bf735c56f43029889df54294c897b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:01.453598 20839 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a4169e5554c4b9c951b8a947ad974c2" candidate_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962bf735c56f43029889df54294c897b" is_pre_election: true
I20251028 21:12:01.453642 20915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a4169e5554c4b9c951b8a947ad974c2" candidate_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600621808894b91a0654602a14118c8" is_pre_election: true
I20251028 21:12:01.453708 20915 raft_consensus.cc:2393] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ec21c1b6e8f4f6f9c4000b92a9bd62d in current term 1: Already voted for candidate 962bf735c56f43029889df54294c897b in this term.
I20251028 21:12:01.453848 20953 leader_election.cc:304] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ec21c1b6e8f4f6f9c4000b92a9bd62d; no voters: 8600621808894b91a0654602a14118c8, 962bf735c56f43029889df54294c897b
I20251028 21:12:01.453919 21047 raft_consensus.cc:2749] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 21:12:01.465438 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:01.466565 21063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:01.466650 21064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:01.466584 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:01.466709 21066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:01.466934 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:01.466969 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:01.466980 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685921466981 us; error 0 us; skew 500 ppm
I20251028 21:12:01.467672 16180 webserver.cc:492] Webserver started at http://127.15.205.4:43757/ using document root <none> and password file <none>
I20251028 21:12:01.467754 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:01.467783 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:01.467820 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:01.468036 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-3-root/instance:
uuid: "08e41de8adad4cc28f1d48ef99ab5651"
format_stamp: "Formatted at 2025-10-28 21:12:01 on dist-test-slave-zmkx"
I20251028 21:12:01.468761 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:01.469133 21071 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.469251 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.469295 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-3-root
uuid: "08e41de8adad4cc28f1d48ef99ab5651"
format_stamp: "Formatted at 2025-10-28 21:12:01 on dist-test-slave-zmkx"
I20251028 21:12:01.469331 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:01.485726 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:01.485992 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:01.486145 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:01.486304 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:01.486339 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.486375 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:01.486402 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:01.489313 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.4:43415
I20251028 21:12:01.489363 21134 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.4:43415 every 8 connection(s)
I20251028 21:12:01.491791 21135 heartbeater.cc:344] Connected to a master server at 127.15.205.62:46785
I20251028 21:12:01.491855 21135 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:01.491940 21135 heartbeater.cc:507] Master 127.15.205.62:46785 requested a full tablet report, sending...
I20251028 21:12:01.492136 20736 ts_manager.cc:194] Registered new tserver with Master: 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4:43415)
I20251028 21:12:01.492394 20736 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52262
I20251028 21:12:01.499651 21048 consensus_queue.cc:1048] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 21:12:01.500432 21058 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20251028 21:12:01.500489 21058 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20251028 21:12:01.500905 21048 consensus_queue.cc:1048] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:01.514351 21038 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:01.525262 21038 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:01.546427 21048 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:01.547600 21048 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:02.283680 20790 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:02.286366 20839 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "d573c542ddc24556a8142e2b2a56f12a"
dest_uuid: "962bf735c56f43029889df54294c897b"
mode: GRACEFUL
new_leader_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d"
from {username='slave'} at 127.0.0.1:53490
I20251028 21:12:02.286478 20839 raft_consensus.cc:606] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Received request to transfer leadership to TS 4ec21c1b6e8f4f6f9c4000b92a9bd62d
I20251028 21:12:02.286635 20790 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251028 21:12:02.286684 20790 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:12:02.286656 20838 consensus_queue.cc:237] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } }
I20251028 21:12:02.287189 20990 raft_consensus.cc:1275] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Refusing update from remote peer 962bf735c56f43029889df54294c897b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:02.287355 20915 raft_consensus.cc:1275] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Refusing update from remote peer 962bf735c56f43029889df54294c897b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:02.287380 21153 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:02.287531 21058 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:02.288101 20915 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } } }
I20251028 21:12:02.288112 20990 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } } }
I20251028 21:12:02.289081 20736 catalog_manager.cc:5649] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d reported cstate change: config changed from index -1 to 2, NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "962bf735c56f43029889df54294c897b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } } }
W20251028 21:12:02.290221 20805 consensus_peers.cc:597] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b -> Peer 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4:43415): Couldn't send request to peer 08e41de8adad4cc28f1d48ef99ab5651. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a4169e5554c4b9c951b8a947ad974c2. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:02.356079 21058 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } } }
I20251028 21:12:02.359681 20915 consensus_queue.cc:237] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } }
I20251028 21:12:02.360097 20838 raft_consensus.cc:1275] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Refusing update from remote peer 8600621808894b91a0654602a14118c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:02.360113 20990 raft_consensus.cc:1275] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Refusing update from remote peer 8600621808894b91a0654602a14118c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:02.360293 21162 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:02.360404 21045 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:02.361032 20990 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } } }
I20251028 21:12:02.361075 20838 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } } }
I20251028 21:12:02.362146 20735 catalog_manager.cc:5649] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b reported cstate change: config changed from index -1 to 2, NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "8600621808894b91a0654602a14118c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } } }
W20251028 21:12:02.363091 20881 consensus_peers.cc:597] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 -> Peer 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4:43415): Couldn't send request to peer 08e41de8adad4cc28f1d48ef99ab5651. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b489548a0414e52b8d8a0af293e9ab1. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:02.371518 21165 ts_tablet_manager.cc:933] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: Initiating tablet copy from peer 962bf735c56f43029889df54294c897b (127.15.205.1:40907)
I20251028 21:12:02.371659 21165 tablet_copy_client.cc:323] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.1:40907
I20251028 21:12:02.373623 20849 tablet_copy_service.cc:140] P 962bf735c56f43029889df54294c897b: Received BeginTabletCopySession request for tablet 9a4169e5554c4b9c951b8a947ad974c2 from peer 08e41de8adad4cc28f1d48ef99ab5651 ({username='slave'} at 127.0.0.1:53500)
I20251028 21:12:02.373672 20849 tablet_copy_service.cc:161] P 962bf735c56f43029889df54294c897b: Beginning new tablet copy session on tablet 9a4169e5554c4b9c951b8a947ad974c2 from peer 08e41de8adad4cc28f1d48ef99ab5651 at {username='slave'} at 127.0.0.1:53500: session id = 08e41de8adad4cc28f1d48ef99ab5651-9a4169e5554c4b9c951b8a947ad974c2
I20251028 21:12:02.374028 20849 tablet_copy_source_session.cc:215] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:02.374313 21165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a4169e5554c4b9c951b8a947ad974c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:02.375072 21165 tablet_copy_client.cc:806] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:02.375187 21165 tablet_copy_client.cc:670] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:02.375706 21165 tablet_copy_client.cc:538] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:02.376204 21165 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: Bootstrap starting.
I20251028 21:12:02.377243 21165 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:02.377389 21165 tablet_bootstrap.cc:492] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: Bootstrap complete.
I20251028 21:12:02.377466 21165 ts_tablet_manager.cc:1403] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:02.377599 21165 raft_consensus.cc:359] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } }
I20251028 21:12:02.377647 21165 raft_consensus.cc:740] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 08e41de8adad4cc28f1d48ef99ab5651, State: Initialized, Role: LEARNER
I20251028 21:12:02.377678 21165 consensus_queue.cc:260] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } }
I20251028 21:12:02.377815 21165 ts_tablet_manager.cc:1434] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:02.377840 21135 heartbeater.cc:499] Master 127.15.205.62:46785 was elected leader, sending a full tablet report...
I20251028 21:12:02.377974 20849 tablet_copy_service.cc:342] P 962bf735c56f43029889df54294c897b: Request end of tablet copy session 08e41de8adad4cc28f1d48ef99ab5651-9a4169e5554c4b9c951b8a947ad974c2 received from {username='slave'} at 127.0.0.1:53500
I20251028 21:12:02.378022 20849 tablet_copy_service.cc:434] P 962bf735c56f43029889df54294c897b: ending tablet copy session 08e41de8adad4cc28f1d48ef99ab5651-9a4169e5554c4b9c951b8a947ad974c2 on tablet 9a4169e5554c4b9c951b8a947ad974c2 with peer 08e41de8adad4cc28f1d48ef99ab5651
I20251028 21:12:02.378724 21153 raft_consensus.cc:993] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b: : Instructing follower 4ec21c1b6e8f4f6f9c4000b92a9bd62d to start an election
I20251028 21:12:02.378778 21153 raft_consensus.cc:1081] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Signalling peer 4ec21c1b6e8f4f6f9c4000b92a9bd62d to start an election
I20251028 21:12:02.378970 20990 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "d573c542ddc24556a8142e2b2a56f12a"
dest_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d"
from {username='slave'} at 127.0.0.1:35914
I20251028 21:12:02.379048 20990 raft_consensus.cc:493] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251028 21:12:02.379074 20990 raft_consensus.cc:3060] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:02.379586 20990 raft_consensus.cc:515] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:02.379698 20990 leader_election.cc:290] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [CANDIDATE]: Term 2 election: Requested vote from peers 962bf735c56f43029889df54294c897b (127.15.205.1:40907), 8600621808894b91a0654602a14118c8 (127.15.205.2:36673)
I20251028 21:12:02.379805 20838 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d573c542ddc24556a8142e2b2a56f12a" candidate_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "962bf735c56f43029889df54294c897b"
I20251028 21:12:02.379863 20838 raft_consensus.cc:3055] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Stepping down as leader of term 1
I20251028 21:12:02.379845 20915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d573c542ddc24556a8142e2b2a56f12a" candidate_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600621808894b91a0654602a14118c8"
I20251028 21:12:02.379904 20838 raft_consensus.cc:740] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 962bf735c56f43029889df54294c897b, State: Running, Role: LEADER
I20251028 21:12:02.379927 20915 raft_consensus.cc:3060] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:02.379966 20838 consensus_queue.cc:260] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:02.380024 20838 raft_consensus.cc:3060] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:02.380380 20915 raft_consensus.cc:2468] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ec21c1b6e8f4f6f9c4000b92a9bd62d in term 2.
I20251028 21:12:02.380445 20838 raft_consensus.cc:2468] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ec21c1b6e8f4f6f9c4000b92a9bd62d in term 2.
I20251028 21:12:02.380523 20956 leader_election.cc:304] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec21c1b6e8f4f6f9c4000b92a9bd62d, 8600621808894b91a0654602a14118c8; no voters:
I20251028 21:12:02.380604 21156 raft_consensus.cc:2804] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:12:02.380678 21156 raft_consensus.cc:697] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 2 LEADER]: Becoming Leader. State: Replica: 4ec21c1b6e8f4f6f9c4000b92a9bd62d, State: Running, Role: LEADER
I20251028 21:12:02.380728 21156 consensus_queue.cc:237] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } }
I20251028 21:12:02.393041 21045 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } } }
I20251028 21:12:02.451190 20736 catalog_manager.cc:5649] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d reported cstate change: term changed from 1 to 2, leader changed from 962bf735c56f43029889df54294c897b (127.15.205.1) to 4ec21c1b6e8f4f6f9c4000b92a9bd62d (127.15.205.3). New cstate: current_term: 2 leader_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:02.479239 21109 raft_consensus.cc:1217] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251028 21:12:02.482348 20915 raft_consensus.cc:1275] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 2 FOLLOWER]: Refusing update from remote peer 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251028 21:12:02.482486 21156 consensus_queue.cc:1048] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:02.485000 21165 ts_tablet_manager.cc:933] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: Initiating tablet copy from peer 8600621808894b91a0654602a14118c8 (127.15.205.2:36673)
I20251028 21:12:02.485167 21165 tablet_copy_client.cc:323] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:36673
I20251028 21:12:02.487989 20925 tablet_copy_service.cc:140] P 8600621808894b91a0654602a14118c8: Received BeginTabletCopySession request for tablet 2b489548a0414e52b8d8a0af293e9ab1 from peer 08e41de8adad4cc28f1d48ef99ab5651 ({username='slave'} at 127.0.0.1:54202)
I20251028 21:12:02.488039 20925 tablet_copy_service.cc:161] P 8600621808894b91a0654602a14118c8: Beginning new tablet copy session on tablet 2b489548a0414e52b8d8a0af293e9ab1 from peer 08e41de8adad4cc28f1d48ef99ab5651 at {username='slave'} at 127.0.0.1:54202: session id = 08e41de8adad4cc28f1d48ef99ab5651-2b489548a0414e52b8d8a0af293e9ab1
I20251028 21:12:02.488592 20925 tablet_copy_source_session.cc:215] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:02.488852 21165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b489548a0414e52b8d8a0af293e9ab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:02.489966 21165 tablet_copy_client.cc:806] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:02.490074 21165 tablet_copy_client.cc:670] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:02.490473 21165 tablet_copy_client.cc:538] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:02.491096 21165 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: Bootstrap starting.
I20251028 21:12:02.492334 21165 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:02.492497 21165 tablet_bootstrap.cc:492] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: Bootstrap complete.
I20251028 21:12:02.492575 21165 ts_tablet_manager.cc:1403] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:02.492704 21165 raft_consensus.cc:359] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } }
I20251028 21:12:02.492803 21165 raft_consensus.cc:740] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 08e41de8adad4cc28f1d48ef99ab5651, State: Initialized, Role: LEARNER
I20251028 21:12:02.492846 21165 consensus_queue.cc:260] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: true } }
I20251028 21:12:02.492962 21165 ts_tablet_manager.cc:1434] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:02.493137 20925 tablet_copy_service.cc:342] P 8600621808894b91a0654602a14118c8: Request end of tablet copy session 08e41de8adad4cc28f1d48ef99ab5651-2b489548a0414e52b8d8a0af293e9ab1 received from {username='slave'} at 127.0.0.1:54202
I20251028 21:12:02.493175 20925 tablet_copy_service.cc:434] P 8600621808894b91a0654602a14118c8: ending tablet copy session 08e41de8adad4cc28f1d48ef99ab5651-2b489548a0414e52b8d8a0af293e9ab1 on tablet 2b489548a0414e52b8d8a0af293e9ab1 with peer 08e41de8adad4cc28f1d48ef99ab5651
I20251028 21:12:02.493409 20839 raft_consensus.cc:1275] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 2 FOLLOWER]: Refusing update from remote peer 4ec21c1b6e8f4f6f9c4000b92a9bd62d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251028 21:12:02.493549 21156 consensus_queue.cc:1048] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [LEADER]: Connected to new peer: Peer: permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:02.572158 21153 raft_consensus.cc:1064] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b: attempting to promote NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 to VOTER
I20251028 21:12:02.572348 21153 consensus_queue.cc:237] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } }
I20251028 21:12:02.572721 21109 raft_consensus.cc:1275] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 LEARNER]: Refusing update from remote peer 962bf735c56f43029889df54294c897b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:02.572759 20915 raft_consensus.cc:1275] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Refusing update from remote peer 962bf735c56f43029889df54294c897b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:02.572875 20990 raft_consensus.cc:1275] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Refusing update from remote peer 962bf735c56f43029889df54294c897b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:02.572906 21058 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:02.572978 21145 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:02.573452 21145 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:02.573951 21109 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.574756 20990 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.575366 20734 catalog_manager.cc:5649] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 reported cstate change: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "962bf735c56f43029889df54294c897b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.596920 21109 raft_consensus.cc:1217] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251028 21:12:02.621745 21058 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.622594 20915 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } attrs { replace: true } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.656394 20838 consensus_queue.cc:237] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } }
I20251028 21:12:02.656786 21109 raft_consensus.cc:1275] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Refusing update from remote peer 962bf735c56f43029889df54294c897b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:02.656808 20990 raft_consensus.cc:1275] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Refusing update from remote peer 962bf735c56f43029889df54294c897b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:02.656989 21179 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:02.657107 21153 consensus_queue.cc:1048] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:02.657747 20990 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8600621808894b91a0654602a14118c8 (127.15.205.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.657758 21109 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8600621808894b91a0654602a14118c8 (127.15.205.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.658754 20733 catalog_manager.cc:5649] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 reported cstate change: config changed from index 3 to 4, VOTER 8600621808894b91a0654602a14118c8 (127.15.205.2) evicted. New cstate: current_term: 1 leader_uuid: "962bf735c56f43029889df54294c897b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.743664 20903 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a4169e5554c4b9c951b8a947ad974c2 with delete_type TABLET_DATA_TOMBSTONED (TS 8600621808894b91a0654602a14118c8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54158
I20251028 21:12:02.743983 21180 tablet_replica.cc:333] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: stopping tablet replica
I20251028 21:12:02.744055 21180 raft_consensus.cc:2243] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:02.744143 21180 raft_consensus.cc:2272] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:02.744424 21180 ts_tablet_manager.cc:1916] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:12:02.746033 21180 ts_tablet_manager.cc:1929] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251028 21:12:02.746101 21180 log.cc:1199] T 9a4169e5554c4b9c951b8a947ad974c2 P 8600621808894b91a0654602a14118c8: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/9a4169e5554c4b9c951b8a947ad974c2
I20251028 21:12:02.746435 20723 catalog_manager.cc:4985] TS 8600621808894b91a0654602a14118c8 (127.15.205.2:36673): tablet 9a4169e5554c4b9c951b8a947ad974c2 (table test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]) successfully deleted
I20251028 21:12:02.791322 21171 raft_consensus.cc:1064] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8: attempting to promote NON_VOTER 08e41de8adad4cc28f1d48ef99ab5651 to VOTER
I20251028 21:12:02.791553 21171 consensus_queue.cc:237] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } }
I20251028 21:12:02.791949 20990 raft_consensus.cc:1275] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Refusing update from remote peer 8600621808894b91a0654602a14118c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:02.791951 21109 raft_consensus.cc:1275] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 LEARNER]: Refusing update from remote peer 8600621808894b91a0654602a14118c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:02.791978 20839 raft_consensus.cc:1275] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Refusing update from remote peer 8600621808894b91a0654602a14118c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:02.792147 21164 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:02.792220 21171 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:02.792294 21162 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:02.793184 21109 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.793259 20839 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.794075 21162 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.794287 20735 catalog_manager.cc:5649] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b reported cstate change: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8600621808894b91a0654602a14118c8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.795043 20990 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 08e41de8adad4cc28f1d48ef99ab5651 (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } attrs { replace: true } } peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.850916 21058 raft_consensus.cc:2955] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8600621808894b91a0654602a14118c8 (127.15.205.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "962bf735c56f43029889df54294c897b" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 40907 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.851820 20720 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 9a4169e5554c4b9c951b8a947ad974c2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:02.860277 20915 consensus_queue.cc:237] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } }
I20251028 21:12:02.860646 20990 raft_consensus.cc:1275] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Refusing update from remote peer 8600621808894b91a0654602a14118c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:02.860695 21109 raft_consensus.cc:1275] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Refusing update from remote peer 8600621808894b91a0654602a14118c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:02.860885 21182 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:02.860993 21171 consensus_queue.cc:1048] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:02.861701 20990 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 962bf735c56f43029889df54294c897b (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.862665 21109 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 962bf735c56f43029889df54294c897b (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.862712 20734 catalog_manager.cc:5649] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d reported cstate change: config changed from index 3 to 4, VOTER 962bf735c56f43029889df54294c897b (127.15.205.1) evicted. New cstate: current_term: 1 leader_uuid: "8600621808894b91a0654602a14118c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.915609 20828 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b489548a0414e52b8d8a0af293e9ab1 with delete_type TABLET_DATA_TOMBSTONED (TS 962bf735c56f43029889df54294c897b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53452
I20251028 21:12:02.915848 21183 tablet_replica.cc:333] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: stopping tablet replica
I20251028 21:12:02.915927 21183 raft_consensus.cc:2243] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:02.915989 21183 raft_consensus.cc:2272] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:02.916217 21183 ts_tablet_manager.cc:1916] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:12:02.917538 21183 ts_tablet_manager.cc:1929] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251028 21:12:02.917608 21183 log.cc:1199] T 2b489548a0414e52b8d8a0af293e9ab1 P 962bf735c56f43029889df54294c897b: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/2b489548a0414e52b8d8a0af293e9ab1
I20251028 21:12:02.917927 20720 catalog_manager.cc:4985] TS 962bf735c56f43029889df54294c897b (127.15.205.1:40907): tablet 2b489548a0414e52b8d8a0af293e9ab1 (table test-workload [id=2a3da42ef3f34d75a727d42ac7fb7735]) successfully deleted
I20251028 21:12:02.991233 21171 raft_consensus.cc:2955] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 962bf735c56f43029889df54294c897b (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8600621808894b91a0654602a14118c8" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 36673 } } peers { permanent_uuid: "4ec21c1b6e8f4f6f9c4000b92a9bd62d" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 42901 } } peers { permanent_uuid: "08e41de8adad4cc28f1d48ef99ab5651" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 43415 } attrs { promote: false } } }
I20251028 21:12:02.992096 20723 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 2b489548a0414e52b8d8a0af293e9ab1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:04.286844 20790 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:04.287034 20790 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:04.287076 20790 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:12:06.287223 20790 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:06.287401 20790 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:06.287441 20790 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:12:08.287595 20790 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:08.287743 20790 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:08.287784 20790 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:12:10.287999 20790 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:10.288144 20790 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:10.288187 20790 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:12:10.539433 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:12:10.541985 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:10.542097 16180 tablet_replica.cc:333] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b: stopping tablet replica
I20251028 21:12:10.542153 16180 raft_consensus.cc:2243] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:10.542215 16180 raft_consensus.cc:2272] T 9a4169e5554c4b9c951b8a947ad974c2 P 962bf735c56f43029889df54294c897b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.542361 16180 tablet_replica.cc:333] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b: stopping tablet replica
I20251028 21:12:10.542385 16180 raft_consensus.cc:2243] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:10.542407 16180 raft_consensus.cc:2272] T d573c542ddc24556a8142e2b2a56f12a P 962bf735c56f43029889df54294c897b [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.554258 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:12:10.555099 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:12:10.557828 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:10.557945 16180 tablet_replica.cc:333] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8: stopping tablet replica
I20251028 21:12:10.558012 16180 raft_consensus.cc:2243] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:10.558068 16180 raft_consensus.cc:2272] T 2b489548a0414e52b8d8a0af293e9ab1 P 8600621808894b91a0654602a14118c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.558190 16180 tablet_replica.cc:333] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8: stopping tablet replica
I20251028 21:12:10.558221 16180 raft_consensus.cc:2243] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:10.558257 16180 raft_consensus.cc:2272] T d573c542ddc24556a8142e2b2a56f12a P 8600621808894b91a0654602a14118c8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.569733 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:12:10.570542 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
W20251028 21:12:10.571208 20953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.205.1:40907: connect: Connection refused (error 111)
W20251028 21:12:10.572198 20953 consensus_peers.cc:597] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d -> Peer 962bf735c56f43029889df54294c897b (127.15.205.1:40907): Couldn't send request to peer 962bf735c56f43029889df54294c897b. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:40907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:10.573443 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:10.573534 16180 tablet_replica.cc:333] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: stopping tablet replica
I20251028 21:12:10.573582 16180 raft_consensus.cc:2243] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:10.573642 16180 raft_consensus.cc:2272] T 2b489548a0414e52b8d8a0af293e9ab1 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.573763 16180 tablet_replica.cc:333] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: stopping tablet replica
I20251028 21:12:10.573792 16180 raft_consensus.cc:2243] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:10.573843 16180 raft_consensus.cc:2272] T 9a4169e5554c4b9c951b8a947ad974c2 P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.573952 16180 tablet_replica.cc:333] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d: stopping tablet replica
I20251028 21:12:10.573978 16180 raft_consensus.cc:2243] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:10.574061 16180 raft_consensus.cc:2272] T d573c542ddc24556a8142e2b2a56f12a P 4ec21c1b6e8f4f6f9c4000b92a9bd62d [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.585371 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:12:10.586179 16180 tablet_server.cc:178] TabletServer@127.15.205.4:0 shutting down...
I20251028 21:12:10.588557 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:10.588641 16180 tablet_replica.cc:333] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651: stopping tablet replica
I20251028 21:12:10.588694 16180 raft_consensus.cc:2243] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:10.588738 16180 raft_consensus.cc:2272] T 2b489548a0414e52b8d8a0af293e9ab1 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.588836 16180 tablet_replica.cc:333] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651: stopping tablet replica
I20251028 21:12:10.588866 16180 raft_consensus.cc:2243] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:10.588894 16180 raft_consensus.cc:2272] T 9a4169e5554c4b9c951b8a947ad974c2 P 08e41de8adad4cc28f1d48ef99ab5651 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.599951 16180 tablet_server.cc:195] TabletServer@127.15.205.4:0 shutdown complete.
I20251028 21:12:10.600590 16180 master.cc:561] Master@127.15.205.62:46785 shutting down...
I20251028 21:12:10.602607 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:10.602655 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:10.602681 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P e09b9fa3084d4b459268d2f0f031480a: stopping tablet replica
I20251028 21:12:10.613979 16180 master.cc:583] Master@127.15.205.62:46785 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingUnstableCluster (10358 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor
I20251028 21:12:10.619010 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:35171
I20251028 21:12:10.619134 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:10.620087 21188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:10.620149 21191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:10.620092 21189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:10.620299 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:10.620404 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:10.620433 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:10.620451 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685930620451 us; error 0 us; skew 500 ppm
I20251028 21:12:10.620913 16180 webserver.cc:492] Webserver started at http://127.15.205.62:35291/ using document root <none> and password file <none>
I20251028 21:12:10.620980 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:10.621021 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:10.621061 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:10.621271 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "2213f38c71d84997b9862d0ac51782ad"
format_stamp: "Formatted at 2025-10-28 21:12:10 on dist-test-slave-zmkx"
I20251028 21:12:10.621984 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:10.622336 21196 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.622442 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.622478 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "2213f38c71d84997b9862d0ac51782ad"
format_stamp: "Formatted at 2025-10-28 21:12:10 on dist-test-slave-zmkx"
I20251028 21:12:10.622517 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:10.630271 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:10.630395 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:10.632498 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:35171
I20251028 21:12:10.635550 21248 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:35171 every 8 connection(s)
I20251028 21:12:10.635663 21249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:10.636494 21249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad: Bootstrap starting.
I20251028 21:12:10.636723 21249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:10.637113 21249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad: No bootstrap required, opened a new log
I20251028 21:12:10.637218 21249 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2213f38c71d84997b9862d0ac51782ad" member_type: VOTER }
I20251028 21:12:10.637257 21249 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:10.637276 21249 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2213f38c71d84997b9862d0ac51782ad, State: Initialized, Role: FOLLOWER
I20251028 21:12:10.637323 21249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2213f38c71d84997b9862d0ac51782ad" member_type: VOTER }
I20251028 21:12:10.637367 21249 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:12:10.637389 21249 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:12:10.637414 21249 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:10.637818 21249 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2213f38c71d84997b9862d0ac51782ad" member_type: VOTER }
I20251028 21:12:10.637867 21249 leader_election.cc:304] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2213f38c71d84997b9862d0ac51782ad; no voters:
I20251028 21:12:10.637955 21249 leader_election.cc:290] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:12:10.637997 21252 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:10.638072 21249 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:12:10.638139 21252 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 1 LEADER]: Becoming Leader. State: Replica: 2213f38c71d84997b9862d0ac51782ad, State: Running, Role: LEADER
I20251028 21:12:10.638191 21252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2213f38c71d84997b9862d0ac51782ad" member_type: VOTER }
I20251028 21:12:10.638366 21253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2213f38c71d84997b9862d0ac51782ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2213f38c71d84997b9862d0ac51782ad" member_type: VOTER } }
I20251028 21:12:10.638382 21254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2213f38c71d84997b9862d0ac51782ad. Latest consensus state: current_term: 1 leader_uuid: "2213f38c71d84997b9862d0ac51782ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2213f38c71d84997b9862d0ac51782ad" member_type: VOTER } }
I20251028 21:12:10.638428 21253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:10.638437 21254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:10.638655 21257 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:12:10.638783 21257 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:12:10.639343 21257 catalog_manager.cc:1357] Generated new cluster ID: 1954bb9614f64a7581595f7a91102835
I20251028 21:12:10.639345 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:12:10.639411 21257 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:12:10.642150 21257 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:12:10.642488 21257 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:12:10.652762 21257 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad: Generated new TSK 0
I20251028 21:12:10.652822 21257 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:12:10.654912 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:10.655856 21271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:10.655915 21270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:10.655964 21273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:10.656082 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:10.656190 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:10.656219 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:10.656240 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685930656240 us; error 0 us; skew 500 ppm
I20251028 21:12:10.656700 16180 webserver.cc:492] Webserver started at http://127.15.205.1:38359/ using document root <none> and password file <none>
I20251028 21:12:10.656773 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:10.656821 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:10.656865 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:10.657076 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "0f658bf1e2424e9caef69948bd2f69af"
format_stamp: "Formatted at 2025-10-28 21:12:10 on dist-test-slave-zmkx"
I20251028 21:12:10.657827 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:10.658193 21278 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.658305 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.658341 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "0f658bf1e2424e9caef69948bd2f69af"
format_stamp: "Formatted at 2025-10-28 21:12:10 on dist-test-slave-zmkx"
I20251028 21:12:10.658380 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:10.678485 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:10.678637 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:10.678740 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:10.678884 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:10.678908 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.678941 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:10.678964 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.681308 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:35339
I20251028 21:12:10.681334 21341 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:35339 every 8 connection(s)
I20251028 21:12:10.681598 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:10.682452 21346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:10.682514 21349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:10.682701 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:10.682780 21347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:10.682941 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:10.682973 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:10.682996 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685930682995 us; error 0 us; skew 500 ppm
I20251028 21:12:10.683460 16180 webserver.cc:492] Webserver started at http://127.15.205.2:43207/ using document root <none> and password file <none>
I20251028 21:12:10.683531 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:10.683569 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:10.683610 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:10.683815 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "7c5f64f5850243d0b59d50297e4108c2"
format_stamp: "Formatted at 2025-10-28 21:12:10 on dist-test-slave-zmkx"
I20251028 21:12:10.684756 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:10.684906 21342 heartbeater.cc:344] Connected to a master server at 127.15.205.62:35171
I20251028 21:12:10.684971 21342 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:10.685071 21342 heartbeater.cc:507] Master 127.15.205.62:35171 requested a full tablet report, sending...
I20251028 21:12:10.685252 21354 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.685295 21213 ts_manager.cc:194] Registered new tserver with Master: 0f658bf1e2424e9caef69948bd2f69af (127.15.205.1:35339)
I20251028 21:12:10.685410 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.685448 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "7c5f64f5850243d0b59d50297e4108c2"
format_stamp: "Formatted at 2025-10-28 21:12:10 on dist-test-slave-zmkx"
I20251028 21:12:10.685485 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:10.686050 21213 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41616
I20251028 21:12:10.715490 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:10.715629 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:10.715721 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:10.715865 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:10.715895 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.715912 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:10.715934 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.718362 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:43067
I20251028 21:12:10.718393 21417 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:43067 every 8 connection(s)
I20251028 21:12:10.718672 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:10.719686 21422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:10.719733 21421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:10.720078 21424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:10.720151 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:10.720358 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:10.720399 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:10.720419 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685930720420 us; error 0 us; skew 500 ppm
I20251028 21:12:10.720968 16180 webserver.cc:492] Webserver started at http://127.15.205.3:46303/ using document root <none> and password file <none>
I20251028 21:12:10.721050 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:10.721092 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:10.721140 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:10.721414 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "52e902ea837e4c259628ec0c90eb6ade"
format_stamp: "Formatted at 2025-10-28 21:12:10 on dist-test-slave-zmkx"
I20251028 21:12:10.721644 21418 heartbeater.cc:344] Connected to a master server at 127.15.205.62:35171
I20251028 21:12:10.721698 21418 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:10.721791 21418 heartbeater.cc:507] Master 127.15.205.62:35171 requested a full tablet report, sending...
I20251028 21:12:10.721984 21213 ts_manager.cc:194] Registered new tserver with Master: 7c5f64f5850243d0b59d50297e4108c2 (127.15.205.2:43067)
I20251028 21:12:10.722316 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:10.722399 21213 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41618
I20251028 21:12:10.722733 21429 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.722832 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.722872 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "52e902ea837e4c259628ec0c90eb6ade"
format_stamp: "Formatted at 2025-10-28 21:12:10 on dist-test-slave-zmkx"
I20251028 21:12:10.722904 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:10.739770 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:10.739893 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:10.739984 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:10.740149 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:10.740173 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.740199 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:10.740227 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:10.742724 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:34147
I20251028 21:12:10.742774 21492 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:34147 every 8 connection(s)
I20251028 21:12:10.745168 21493 heartbeater.cc:344] Connected to a master server at 127.15.205.62:35171
I20251028 21:12:10.745220 21493 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:10.745311 21493 heartbeater.cc:507] Master 127.15.205.62:35171 requested a full tablet report, sending...
I20251028 21:12:10.745483 21213 ts_manager.cc:194] Registered new tserver with Master: 52e902ea837e4c259628ec0c90eb6ade (127.15.205.3:34147)
I20251028 21:12:10.745749 21213 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41632
I20251028 21:12:10.746152 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003274866s
I20251028 21:12:11.686465 21342 heartbeater.cc:499] Master 127.15.205.62:35171 was elected leader, sending a full tablet report...
I20251028 21:12:11.722862 21418 heartbeater.cc:499] Master 127.15.205.62:35171 was elected leader, sending a full tablet report...
I20251028 21:12:11.746206 21493 heartbeater.cc:499] Master 127.15.205.62:35171 was elected leader, sending a full tablet report...
I20251028 21:12:11.771330 16180 test_util.cc:276] Using random seed: 2016101825
I20251028 21:12:11.775542 21213 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41640:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:12:11.775660 21213 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:12:11.779841 21382 tablet_service.cc:1505] Processing CreateTablet for tablet 53b6c536cb9944ab8c9b898450effe95 (DEFAULT_TABLE table=test-workload [id=9a504a5410f04cbf9a787f3c31c4684b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:12:11.779930 21306 tablet_service.cc:1505] Processing CreateTablet for tablet 53b6c536cb9944ab8c9b898450effe95 (DEFAULT_TABLE table=test-workload [id=9a504a5410f04cbf9a787f3c31c4684b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:12:11.779945 21457 tablet_service.cc:1505] Processing CreateTablet for tablet 53b6c536cb9944ab8c9b898450effe95 (DEFAULT_TABLE table=test-workload [id=9a504a5410f04cbf9a787f3c31c4684b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 21:12:11.779976 21382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b6c536cb9944ab8c9b898450effe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:11.780092 21306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b6c536cb9944ab8c9b898450effe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:11.780092 21457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b6c536cb9944ab8c9b898450effe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:11.781066 21513 tablet_bootstrap.cc:492] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2: Bootstrap starting.
I20251028 21:12:11.781394 21515 tablet_bootstrap.cc:492] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af: Bootstrap starting.
I20251028 21:12:11.781462 21513 tablet_bootstrap.cc:654] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:11.781494 21514 tablet_bootstrap.cc:492] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade: Bootstrap starting.
I20251028 21:12:11.781899 21515 tablet_bootstrap.cc:654] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:11.782208 21514 tablet_bootstrap.cc:654] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:11.782394 21515 tablet_bootstrap.cc:492] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af: No bootstrap required, opened a new log
I20251028 21:12:11.782461 21515 ts_tablet_manager.cc:1403] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:11.782590 21513 tablet_bootstrap.cc:492] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2: No bootstrap required, opened a new log
I20251028 21:12:11.782632 21513 ts_tablet_manager.cc:1403] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:11.782608 21515 raft_consensus.cc:359] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:11.782688 21515 raft_consensus.cc:385] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:11.782719 21515 raft_consensus.cc:740] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f658bf1e2424e9caef69948bd2f69af, State: Initialized, Role: FOLLOWER
I20251028 21:12:11.782780 21513 raft_consensus.cc:359] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:11.782783 21515 consensus_queue.cc:260] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:11.782837 21513 raft_consensus.cc:385] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:11.782868 21513 raft_consensus.cc:740] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c5f64f5850243d0b59d50297e4108c2, State: Initialized, Role: FOLLOWER
I20251028 21:12:11.782945 21513 consensus_queue.cc:260] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:11.782965 21515 ts_tablet_manager.cc:1434] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:11.783080 21513 ts_tablet_manager.cc:1434] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:11.783377 21514 tablet_bootstrap.cc:492] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade: No bootstrap required, opened a new log
I20251028 21:12:11.783455 21514 ts_tablet_manager.cc:1403] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:11.783591 21514 raft_consensus.cc:359] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:11.783654 21514 raft_consensus.cc:385] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:11.783684 21514 raft_consensus.cc:740] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52e902ea837e4c259628ec0c90eb6ade, State: Initialized, Role: FOLLOWER
I20251028 21:12:11.783730 21514 consensus_queue.cc:260] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:11.783854 21514 ts_tablet_manager.cc:1434] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20251028 21:12:11.932548 21343 tablet.cc:2378] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251028 21:12:12.042481 21521 raft_consensus.cc:493] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:12.042577 21521 raft_consensus.cc:515] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:12.042744 21521 leader_election.cc:290] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f658bf1e2424e9caef69948bd2f69af (127.15.205.1:35339), 7c5f64f5850243d0b59d50297e4108c2 (127.15.205.2:43067)
I20251028 21:12:12.045275 21316 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "53b6c536cb9944ab8c9b898450effe95" candidate_uuid: "52e902ea837e4c259628ec0c90eb6ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f658bf1e2424e9caef69948bd2f69af" is_pre_election: true
I20251028 21:12:12.045352 21316 raft_consensus.cc:2468] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52e902ea837e4c259628ec0c90eb6ade in term 0.
I20251028 21:12:12.045280 21392 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "53b6c536cb9944ab8c9b898450effe95" candidate_uuid: "52e902ea837e4c259628ec0c90eb6ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c5f64f5850243d0b59d50297e4108c2" is_pre_election: true
I20251028 21:12:12.045408 21392 raft_consensus.cc:2468] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52e902ea837e4c259628ec0c90eb6ade in term 0.
I20251028 21:12:12.045485 21430 leader_election.cc:304] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f658bf1e2424e9caef69948bd2f69af, 52e902ea837e4c259628ec0c90eb6ade; no voters:
I20251028 21:12:12.045591 21521 raft_consensus.cc:2804] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:12.045636 21521 raft_consensus.cc:493] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:12.045651 21521 raft_consensus.cc:3060] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:12.046273 21521 raft_consensus.cc:515] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:12.046382 21521 leader_election.cc:290] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [CANDIDATE]: Term 1 election: Requested vote from peers 0f658bf1e2424e9caef69948bd2f69af (127.15.205.1:35339), 7c5f64f5850243d0b59d50297e4108c2 (127.15.205.2:43067)
I20251028 21:12:12.046517 21316 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "53b6c536cb9944ab8c9b898450effe95" candidate_uuid: "52e902ea837e4c259628ec0c90eb6ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f658bf1e2424e9caef69948bd2f69af"
I20251028 21:12:12.046551 21392 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "53b6c536cb9944ab8c9b898450effe95" candidate_uuid: "52e902ea837e4c259628ec0c90eb6ade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c5f64f5850243d0b59d50297e4108c2"
I20251028 21:12:12.046586 21316 raft_consensus.cc:3060] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:12.046592 21392 raft_consensus.cc:3060] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:12.047329 21316 raft_consensus.cc:2468] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52e902ea837e4c259628ec0c90eb6ade in term 1.
I20251028 21:12:12.047334 21392 raft_consensus.cc:2468] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52e902ea837e4c259628ec0c90eb6ade in term 1.
I20251028 21:12:12.047497 21430 leader_election.cc:304] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f658bf1e2424e9caef69948bd2f69af, 52e902ea837e4c259628ec0c90eb6ade; no voters:
I20251028 21:12:12.047590 21521 raft_consensus.cc:2804] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:12.047712 21521 raft_consensus.cc:697] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 1 LEADER]: Becoming Leader. State: Replica: 52e902ea837e4c259628ec0c90eb6ade, State: Running, Role: LEADER
I20251028 21:12:12.047760 21521 consensus_queue.cc:237] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } }
I20251028 21:12:12.048257 21212 catalog_manager.cc:5649] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade reported cstate change: term changed from 0 to 1, leader changed from <none> to 52e902ea837e4c259628ec0c90eb6ade (127.15.205.3). New cstate: current_term: 1 leader_uuid: "52e902ea837e4c259628ec0c90eb6ade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f658bf1e2424e9caef69948bd2f69af" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 35339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52e902ea837e4c259628ec0c90eb6ade" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 34147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:12.058429 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:12:12.060671 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:12.060803 16180 tablet_replica.cc:333] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af: stopping tablet replica
I20251028 21:12:12.060858 16180 raft_consensus.cc:2243] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:12.060897 16180 raft_consensus.cc:2272] T 53b6c536cb9944ab8c9b898450effe95 P 0f658bf1e2424e9caef69948bd2f69af [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:12.072300 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:12:12.513978 21521 consensus_queue.cc:1048] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c5f64f5850243d0b59d50297e4108c2" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 21:12:12.515609 21430 consensus_peers.cc:597] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade -> Peer 0f658bf1e2424e9caef69948bd2f69af (127.15.205.1:35339): Couldn't send request to peer 0f658bf1e2424e9caef69948bd2f69af. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:35339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:12.639394 21268 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:12.639501 21268 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:12.639539 21268 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:12:14.639668 21268 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:14.639782 21268 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:14.639820 21268 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251028 21:12:14.937937 21430 consensus_peers.cc:597] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade -> Peer 0f658bf1e2424e9caef69948bd2f69af (127.15.205.1:35339): Couldn't send request to peer 0f658bf1e2424e9caef69948bd2f69af. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:35339: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251028 21:12:16.024566 21430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.205.1:35339: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20251028 21:12:16.639950 21268 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:16.640074 21268 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:16.640115 21268 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251028 21:12:17.468897 21430 consensus_peers.cc:597] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade -> Peer 0f658bf1e2424e9caef69948bd2f69af (127.15.205.1:35339): Couldn't send request to peer 0f658bf1e2424e9caef69948bd2f69af. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:35339: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20251028 21:12:18.640252 21268 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:18.640368 21268 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:18.640404 21268 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251028 21:12:20.064855 21430 consensus_peers.cc:597] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade -> Peer 0f658bf1e2424e9caef69948bd2f69af (127.15.205.1:35339): Couldn't send request to peer 0f658bf1e2424e9caef69948bd2f69af. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:35339: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251028 21:12:20.640537 21268 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:20.640654 21268 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:20.640671 21268 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251028 21:12:21.433030 21430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.205.1:35339: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20251028 21:12:22.519507 21430 consensus_peers.cc:597] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade -> Peer 0f658bf1e2424e9caef69948bd2f69af (127.15.205.1:35339): Couldn't send request to peer 0f658bf1e2424e9caef69948bd2f69af. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:35339: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20251028 21:12:22.640801 21268 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251028 21:12:22.640930 21268 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251028 21:12:22.640956 21268 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251028 21:12:23.123809 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:12:23.126261 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:23.126387 16180 tablet_replica.cc:333] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2: stopping tablet replica
I20251028 21:12:23.126441 16180 raft_consensus.cc:2243] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:23.126489 16180 raft_consensus.cc:2272] T 53b6c536cb9944ab8c9b898450effe95 P 7c5f64f5850243d0b59d50297e4108c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:23.137972 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:12:23.138738 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:12:23.141650 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:23.141739 16180 tablet_replica.cc:333] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade: stopping tablet replica
I20251028 21:12:23.141798 16180 raft_consensus.cc:2243] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:23.141860 16180 raft_consensus.cc:2272] T 53b6c536cb9944ab8c9b898450effe95 P 52e902ea837e4c259628ec0c90eb6ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:23.153499 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:12:23.154181 16180 master.cc:561] Master@127.15.205.62:35171 shutting down...
I20251028 21:12:23.156271 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:23.156333 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:23.156380 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2213f38c71d84997b9862d0ac51782ad: stopping tablet replica
I20251028 21:12:23.167763 16180 master.cc:583] Master@127.15.205.62:35171 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor (12552 ms)
[ RUN ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering
I20251028 21:12:23.171512 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:34285
I20251028 21:12:23.171648 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:23.172861 21555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:23.172925 21558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:23.172966 21556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:23.173070 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:23.173214 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:23.173249 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:23.173260 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685943173261 us; error 0 us; skew 500 ppm
I20251028 21:12:23.173761 16180 webserver.cc:492] Webserver started at http://127.15.205.62:36655/ using document root <none> and password file <none>
I20251028 21:12:23.173817 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:23.173847 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:23.173880 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:23.174078 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "d06532e233524eccb24a5a7eef46d3b2"
format_stamp: "Formatted at 2025-10-28 21:12:23 on dist-test-slave-zmkx"
I20251028 21:12:23.174831 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:23.175182 21563 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.175364 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.175416 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "d06532e233524eccb24a5a7eef46d3b2"
format_stamp: "Formatted at 2025-10-28 21:12:23 on dist-test-slave-zmkx"
I20251028 21:12:23.175451 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:23.180172 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:23.180292 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:23.182348 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:34285
I20251028 21:12:23.184269 21615 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:34285 every 8 connection(s)
I20251028 21:12:23.184357 21616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:23.185212 21616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2: Bootstrap starting.
I20251028 21:12:23.185457 21616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:23.185853 21616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2: No bootstrap required, opened a new log
I20251028 21:12:23.185962 21616 raft_consensus.cc:359] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06532e233524eccb24a5a7eef46d3b2" member_type: VOTER }
I20251028 21:12:23.186002 21616 raft_consensus.cc:385] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:23.186017 21616 raft_consensus.cc:740] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06532e233524eccb24a5a7eef46d3b2, State: Initialized, Role: FOLLOWER
I20251028 21:12:23.186049 21616 consensus_queue.cc:260] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06532e233524eccb24a5a7eef46d3b2" member_type: VOTER }
I20251028 21:12:23.186096 21616 raft_consensus.cc:399] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:12:23.186115 21616 raft_consensus.cc:493] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:12:23.186136 21616 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:23.186558 21616 raft_consensus.cc:515] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06532e233524eccb24a5a7eef46d3b2" member_type: VOTER }
I20251028 21:12:23.186605 21616 leader_election.cc:304] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d06532e233524eccb24a5a7eef46d3b2; no voters:
I20251028 21:12:23.186689 21616 leader_election.cc:290] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:12:23.186753 21619 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:23.186798 21616 sys_catalog.cc:565] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:12:23.186873 21619 raft_consensus.cc:697] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 1 LEADER]: Becoming Leader. State: Replica: d06532e233524eccb24a5a7eef46d3b2, State: Running, Role: LEADER
I20251028 21:12:23.186923 21619 consensus_queue.cc:237] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06532e233524eccb24a5a7eef46d3b2" member_type: VOTER }
I20251028 21:12:23.187098 21619 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d06532e233524eccb24a5a7eef46d3b2. Latest consensus state: current_term: 1 leader_uuid: "d06532e233524eccb24a5a7eef46d3b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06532e233524eccb24a5a7eef46d3b2" member_type: VOTER } }
I20251028 21:12:23.187130 21621 sys_catalog.cc:455] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d06532e233524eccb24a5a7eef46d3b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d06532e233524eccb24a5a7eef46d3b2" member_type: VOTER } }
I20251028 21:12:23.187232 21619 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:23.187522 21621 sys_catalog.cc:458] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:23.187749 21628 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:12:23.187865 21628 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:12:23.188102 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:12:23.188400 21628 catalog_manager.cc:1357] Generated new cluster ID: d551835507d04434b8f032a597a7d454
I20251028 21:12:23.188448 21628 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:12:23.200348 21628 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:12:23.200695 21628 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:12:23.208811 21628 catalog_manager.cc:6022] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2: Generated new TSK 0
I20251028 21:12:23.208870 21628 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:12:23.219899 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:23.221177 21640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:23.221175 21637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:23.221302 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:23.221177 21638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:23.221431 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:23.221467 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:23.221494 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685943221494 us; error 0 us; skew 500 ppm
I20251028 21:12:23.222074 16180 webserver.cc:492] Webserver started at http://127.15.205.1:35579/ using document root <none> and password file <none>
I20251028 21:12:23.222157 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:23.222206 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:23.222255 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:23.222502 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "bf50c9d915ce4fa8b5f425bc1a260257"
format_stamp: "Formatted at 2025-10-28 21:12:23 on dist-test-slave-zmkx"
I20251028 21:12:23.223359 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:23.223762 21645 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.223878 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.223923 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "bf50c9d915ce4fa8b5f425bc1a260257"
format_stamp: "Formatted at 2025-10-28 21:12:23 on dist-test-slave-zmkx"
I20251028 21:12:23.223961 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:23.254220 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:23.254372 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:23.254501 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:23.254670 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:23.254698 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.254717 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:23.254745 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.257503 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:42845
I20251028 21:12:23.257545 21708 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:42845 every 8 connection(s)
I20251028 21:12:23.257853 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:23.258946 21713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:23.259043 21716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:23.259058 21714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:23.259145 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:23.259395 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:23.259434 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:23.259455 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685943259455 us; error 0 us; skew 500 ppm
I20251028 21:12:23.260064 16180 webserver.cc:492] Webserver started at http://127.15.205.2:39777/ using document root <none> and password file <none>
I20251028 21:12:23.260149 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:23.260190 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:23.260224 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:23.260483 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "d71232d8e053429d9fc30887e484600e"
format_stamp: "Formatted at 2025-10-28 21:12:23 on dist-test-slave-zmkx"
I20251028 21:12:23.261322 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:23.261693 21721 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.261806 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.261847 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "d71232d8e053429d9fc30887e484600e"
format_stamp: "Formatted at 2025-10-28 21:12:23 on dist-test-slave-zmkx"
I20251028 21:12:23.261890 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:23.264250 21709 heartbeater.cc:344] Connected to a master server at 127.15.205.62:34285
I20251028 21:12:23.264308 21709 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:23.264400 21709 heartbeater.cc:507] Master 127.15.205.62:34285 requested a full tablet report, sending...
I20251028 21:12:23.264628 21580 ts_manager.cc:194] Registered new tserver with Master: bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845)
I20251028 21:12:23.265129 21580 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49266
I20251028 21:12:23.270474 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:23.270632 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:23.270742 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:23.270905 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:23.270929 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.270957 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:23.270983 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.273559 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:40967
I20251028 21:12:23.273587 21784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:40967 every 8 connection(s)
I20251028 21:12:23.273871 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:23.274909 21791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:23.274951 21789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:23.275126 21788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:23.275206 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:23.275405 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:23.275441 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:23.275458 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685943275459 us; error 0 us; skew 500 ppm
I20251028 21:12:23.276036 16180 webserver.cc:492] Webserver started at http://127.15.205.3:39437/ using document root <none> and password file <none>
I20251028 21:12:23.276124 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:23.276175 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:23.276219 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:23.276458 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "5989efab16fb4953a4c614f2a38f35f3"
format_stamp: "Formatted at 2025-10-28 21:12:23 on dist-test-slave-zmkx"
I20251028 21:12:23.276798 21785 heartbeater.cc:344] Connected to a master server at 127.15.205.62:34285
I20251028 21:12:23.276863 21785 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:23.276928 21785 heartbeater.cc:507] Master 127.15.205.62:34285 requested a full tablet report, sending...
I20251028 21:12:23.277123 21580 ts_manager.cc:194] Registered new tserver with Master: d71232d8e053429d9fc30887e484600e (127.15.205.2:40967)
I20251028 21:12:23.277382 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:23.277432 21580 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49268
I20251028 21:12:23.277860 21796 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.277982 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.278030 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "5989efab16fb4953a4c614f2a38f35f3"
format_stamp: "Formatted at 2025-10-28 21:12:23 on dist-test-slave-zmkx"
I20251028 21:12:23.278074 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:23.290711 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:23.290905 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:23.291038 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:23.291194 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:23.291218 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.291246 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:23.291291 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:23.294109 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:44093
I20251028 21:12:23.294168 21859 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:44093 every 8 connection(s)
I20251028 21:12:23.296643 21860 heartbeater.cc:344] Connected to a master server at 127.15.205.62:34285
I20251028 21:12:23.296691 21860 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:23.296769 21860 heartbeater.cc:507] Master 127.15.205.62:34285 requested a full tablet report, sending...
I20251028 21:12:23.297155 21580 ts_manager.cc:194] Registered new tserver with Master: 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:23.297499 21580 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49284
I20251028 21:12:23.297618 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003358898s
I20251028 21:12:24.265602 21709 heartbeater.cc:499] Master 127.15.205.62:34285 was elected leader, sending a full tablet report...
I20251028 21:12:24.277935 21785 heartbeater.cc:499] Master 127.15.205.62:34285 was elected leader, sending a full tablet report...
I20251028 21:12:24.298137 21860 heartbeater.cc:499] Master 127.15.205.62:34285 was elected leader, sending a full tablet report...
I20251028 21:12:24.322815 16180 test_util.cc:276] Using random seed: 2028653309
I20251028 21:12:24.327572 21580 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49300:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:12:24.327708 21580 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:12:24.332419 21749 tablet_service.cc:1505] Processing CreateTablet for tablet 5432e30e29a245dda2ab91568fdfef56 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:24.332518 21747 tablet_service.cc:1505] Processing CreateTablet for tablet 449445589fd747a4bd82016e9e3c7882 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:24.332563 21749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5432e30e29a245dda2ab91568fdfef56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.332644 21747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449445589fd747a4bd82016e9e3c7882. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.333144 21673 tablet_service.cc:1505] Processing CreateTablet for tablet 5432e30e29a245dda2ab91568fdfef56 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:24.333253 21673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5432e30e29a245dda2ab91568fdfef56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.333933 21880 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e: Bootstrap starting.
I20251028 21:12:24.334117 21824 tablet_service.cc:1505] Processing CreateTablet for tablet 5432e30e29a245dda2ab91568fdfef56 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:24.334131 21821 tablet_service.cc:1505] Processing CreateTablet for tablet 760984c8fffe45738de29b60bf9a1a4e (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:24.334223 21824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5432e30e29a245dda2ab91568fdfef56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.334326 21821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 760984c8fffe45738de29b60bf9a1a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.334369 21880 tablet_bootstrap.cc:654] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.334456 21672 tablet_service.cc:1505] Processing CreateTablet for tablet fa0913e597d64739b6233bf273d66833 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:24.334554 21672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0913e597d64739b6233bf273d66833. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.335378 21823 tablet_service.cc:1505] Processing CreateTablet for tablet fa0913e597d64739b6233bf273d66833 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:24.335422 21822 tablet_service.cc:1505] Processing CreateTablet for tablet 449445589fd747a4bd82016e9e3c7882 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:24.335490 21823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0913e597d64739b6233bf273d66833. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.333950 21746 tablet_service.cc:1505] Processing CreateTablet for tablet 760984c8fffe45738de29b60bf9a1a4e (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:24.335736 21746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 760984c8fffe45738de29b60bf9a1a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.335765 21671 tablet_service.cc:1505] Processing CreateTablet for tablet 449445589fd747a4bd82016e9e3c7882 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:24.335848 21671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449445589fd747a4bd82016e9e3c7882. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.335345 21883 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3: Bootstrap starting.
I20251028 21:12:24.336759 21670 tablet_service.cc:1505] Processing CreateTablet for tablet 760984c8fffe45738de29b60bf9a1a4e (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:24.336830 21670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 760984c8fffe45738de29b60bf9a1a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.332435 21748 tablet_service.cc:1505] Processing CreateTablet for tablet fa0913e597d64739b6233bf273d66833 (DEFAULT_TABLE table=test-workload [id=c768b61fbd064446960633b581fd1bb4]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:24.337096 21748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0913e597d64739b6233bf273d66833. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.337126 21883 tablet_bootstrap.cc:654] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.337677 21883 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3: No bootstrap required, opened a new log
I20251028 21:12:24.337718 21883 ts_tablet_manager.cc:1403] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:24.337850 21883 raft_consensus.cc:359] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.337898 21883 raft_consensus.cc:385] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.337915 21883 raft_consensus.cc:740] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5989efab16fb4953a4c614f2a38f35f3, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.337975 21883 consensus_queue.cc:260] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.338126 21883 ts_tablet_manager.cc:1434] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:24.338191 21883 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3: Bootstrap starting.
I20251028 21:12:24.338228 21822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449445589fd747a4bd82016e9e3c7882. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:24.338521 21883 tablet_bootstrap.cc:654] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.339010 21883 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3: No bootstrap required, opened a new log
I20251028 21:12:24.339044 21883 ts_tablet_manager.cc:1403] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.339166 21883 raft_consensus.cc:359] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.339213 21883 raft_consensus.cc:385] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.339229 21883 raft_consensus.cc:740] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5989efab16fb4953a4c614f2a38f35f3, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.339272 21883 consensus_queue.cc:260] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.339392 21883 ts_tablet_manager.cc:1434] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:24.339434 21883 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3: Bootstrap starting.
I20251028 21:12:24.339733 21883 tablet_bootstrap.cc:654] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.339781 21881 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257: Bootstrap starting.
I20251028 21:12:24.340190 21881 tablet_bootstrap.cc:654] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.340215 21883 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3: No bootstrap required, opened a new log
I20251028 21:12:24.340252 21883 ts_tablet_manager.cc:1403] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:24.340369 21883 raft_consensus.cc:359] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.340409 21883 raft_consensus.cc:385] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.340431 21883 raft_consensus.cc:740] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5989efab16fb4953a4c614f2a38f35f3, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.340497 21883 consensus_queue.cc:260] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.340713 21880 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e: No bootstrap required, opened a new log
I20251028 21:12:24.340749 21881 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257: No bootstrap required, opened a new log
I20251028 21:12:24.340791 21880 ts_tablet_manager.cc:1403] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20251028 21:12:24.340814 21881 ts_tablet_manager.cc:1403] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.340938 21880 raft_consensus.cc:359] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.341019 21883 ts_tablet_manager.cc:1434] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:24.341002 21880 raft_consensus.cc:385] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.340996 21881 raft_consensus.cc:359] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.341117 21883 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3: Bootstrap starting.
I20251028 21:12:24.341189 21881 raft_consensus.cc:385] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.341156 21880 raft_consensus.cc:740] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71232d8e053429d9fc30887e484600e, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.341240 21881 raft_consensus.cc:740] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf50c9d915ce4fa8b5f425bc1a260257, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.341378 21881 consensus_queue.cc:260] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.341385 21880 consensus_queue.cc:260] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.341543 21881 ts_tablet_manager.cc:1434] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.341559 21883 tablet_bootstrap.cc:654] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.341562 21880 ts_tablet_manager.cc:1434] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:24.341609 21881 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257: Bootstrap starting.
I20251028 21:12:24.341777 21880 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e: Bootstrap starting.
I20251028 21:12:24.342006 21881 tablet_bootstrap.cc:654] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.342161 21880 tablet_bootstrap.cc:654] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.342711 21880 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e: No bootstrap required, opened a new log
I20251028 21:12:24.342756 21880 ts_tablet_manager.cc:1403] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.342805 21881 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257: No bootstrap required, opened a new log
I20251028 21:12:24.342851 21881 ts_tablet_manager.cc:1403] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.342883 21880 raft_consensus.cc:359] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.342936 21880 raft_consensus.cc:385] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.342962 21880 raft_consensus.cc:740] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71232d8e053429d9fc30887e484600e, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.342986 21881 raft_consensus.cc:359] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.343011 21880 consensus_queue.cc:260] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.343040 21881 raft_consensus.cc:385] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.343070 21881 raft_consensus.cc:740] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf50c9d915ce4fa8b5f425bc1a260257, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.343084 21880 ts_tablet_manager.cc:1434] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:24.343114 21881 consensus_queue.cc:260] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.343132 21880 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e: Bootstrap starting.
I20251028 21:12:24.343200 21881 ts_tablet_manager.cc:1434] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:24.343279 21881 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257: Bootstrap starting.
I20251028 21:12:24.343411 21883 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3: No bootstrap required, opened a new log
I20251028 21:12:24.343446 21883 ts_tablet_manager.cc:1403] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:12:24.343550 21880 tablet_bootstrap.cc:654] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.343560 21883 raft_consensus.cc:359] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.343623 21883 raft_consensus.cc:385] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.343645 21883 raft_consensus.cc:740] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5989efab16fb4953a4c614f2a38f35f3, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.343659 21881 tablet_bootstrap.cc:654] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.343691 21883 consensus_queue.cc:260] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.343793 21883 ts_tablet_manager.cc:1434] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:24.344139 21880 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e: No bootstrap required, opened a new log
I20251028 21:12:24.344195 21880 ts_tablet_manager.cc:1403] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.344303 21881 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257: No bootstrap required, opened a new log
I20251028 21:12:24.344348 21881 ts_tablet_manager.cc:1403] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.344314 21880 raft_consensus.cc:359] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.344437 21880 raft_consensus.cc:385] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.344458 21880 raft_consensus.cc:740] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71232d8e053429d9fc30887e484600e, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.344487 21880 consensus_queue.cc:260] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.344494 21881 raft_consensus.cc:359] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.344570 21881 raft_consensus.cc:385] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.344587 21881 raft_consensus.cc:740] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf50c9d915ce4fa8b5f425bc1a260257, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.344590 21880 ts_tablet_manager.cc:1434] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:24.344676 21880 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e: Bootstrap starting.
I20251028 21:12:24.344662 21881 consensus_queue.cc:260] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.344732 21881 ts_tablet_manager.cc:1434] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:24.344772 21881 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257: Bootstrap starting.
I20251028 21:12:24.345060 21880 tablet_bootstrap.cc:654] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.345157 21881 tablet_bootstrap.cc:654] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:24.345618 21880 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e: No bootstrap required, opened a new log
I20251028 21:12:24.345654 21881 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257: No bootstrap required, opened a new log
I20251028 21:12:24.345667 21880 ts_tablet_manager.cc:1403] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.345686 21881 ts_tablet_manager.cc:1403] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:24.345805 21880 raft_consensus.cc:359] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.345821 21881 raft_consensus.cc:359] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.345851 21880 raft_consensus.cc:385] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.345857 21881 raft_consensus.cc:385] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:24.345903 21880 raft_consensus.cc:740] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71232d8e053429d9fc30887e484600e, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.345937 21881 raft_consensus.cc:740] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf50c9d915ce4fa8b5f425bc1a260257, State: Initialized, Role: FOLLOWER
I20251028 21:12:24.345988 21880 consensus_queue.cc:260] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.346006 21881 consensus_queue.cc:260] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.346076 21880 ts_tablet_manager.cc:1434] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:24.346086 21881 ts_tablet_manager.cc:1434] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:24.377597 21890 raft_consensus.cc:493] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:24.377673 21890 raft_consensus.cc:515] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.377867 21890 leader_election.cc:290] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845), 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:24.380414 21834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "449445589fd747a4bd82016e9e3c7882" candidate_uuid: "d71232d8e053429d9fc30887e484600e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5989efab16fb4953a4c614f2a38f35f3" is_pre_election: true
I20251028 21:12:24.380426 21683 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "449445589fd747a4bd82016e9e3c7882" candidate_uuid: "d71232d8e053429d9fc30887e484600e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" is_pre_election: true
I20251028 21:12:24.380491 21834 raft_consensus.cc:2468] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d71232d8e053429d9fc30887e484600e in term 0.
I20251028 21:12:24.380502 21683 raft_consensus.cc:2468] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d71232d8e053429d9fc30887e484600e in term 0.
I20251028 21:12:24.380661 21723 leader_election.cc:304] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf50c9d915ce4fa8b5f425bc1a260257, d71232d8e053429d9fc30887e484600e; no voters:
I20251028 21:12:24.380766 21890 raft_consensus.cc:2804] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:24.380807 21890 raft_consensus.cc:493] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:24.380829 21890 raft_consensus.cc:3060] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.381425 21890 raft_consensus.cc:515] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.381548 21890 leader_election.cc:290] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 1 election: Requested vote from peers bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845), 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:24.381705 21834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "449445589fd747a4bd82016e9e3c7882" candidate_uuid: "d71232d8e053429d9fc30887e484600e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5989efab16fb4953a4c614f2a38f35f3"
I20251028 21:12:24.381735 21683 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "449445589fd747a4bd82016e9e3c7882" candidate_uuid: "d71232d8e053429d9fc30887e484600e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf50c9d915ce4fa8b5f425bc1a260257"
I20251028 21:12:24.381767 21834 raft_consensus.cc:3060] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.381784 21683 raft_consensus.cc:3060] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.382400 21683 raft_consensus.cc:2468] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d71232d8e053429d9fc30887e484600e in term 1.
I20251028 21:12:24.382404 21834 raft_consensus.cc:2468] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d71232d8e053429d9fc30887e484600e in term 1.
I20251028 21:12:24.382553 21723 leader_election.cc:304] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf50c9d915ce4fa8b5f425bc1a260257, d71232d8e053429d9fc30887e484600e; no voters:
I20251028 21:12:24.382630 21890 raft_consensus.cc:2804] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:24.382752 21890 raft_consensus.cc:697] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 1 LEADER]: Becoming Leader. State: Replica: d71232d8e053429d9fc30887e484600e, State: Running, Role: LEADER
I20251028 21:12:24.382791 21890 consensus_queue.cc:237] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.383288 21579 catalog_manager.cc:5649] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e reported cstate change: term changed from 0 to 1, leader changed from <none> to d71232d8e053429d9fc30887e484600e (127.15.205.2). New cstate: current_term: 1 leader_uuid: "d71232d8e053429d9fc30887e484600e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:24.384657 21885 raft_consensus.cc:493] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:24.384702 21885 raft_consensus.cc:515] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.384829 21885 leader_election.cc:290] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d71232d8e053429d9fc30887e484600e (127.15.205.2:40967), bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845)
I20251028 21:12:24.387125 21683 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5432e30e29a245dda2ab91568fdfef56" candidate_uuid: "5989efab16fb4953a4c614f2a38f35f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" is_pre_election: true
I20251028 21:12:24.387204 21683 raft_consensus.cc:2468] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5989efab16fb4953a4c614f2a38f35f3 in term 0.
I20251028 21:12:24.387202 21759 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5432e30e29a245dda2ab91568fdfef56" candidate_uuid: "5989efab16fb4953a4c614f2a38f35f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71232d8e053429d9fc30887e484600e" is_pre_election: true
I20251028 21:12:24.387337 21759 raft_consensus.cc:2468] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5989efab16fb4953a4c614f2a38f35f3 in term 0.
I20251028 21:12:24.387423 21798 leader_election.cc:304] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5989efab16fb4953a4c614f2a38f35f3, bf50c9d915ce4fa8b5f425bc1a260257; no voters:
I20251028 21:12:24.387516 21885 raft_consensus.cc:2804] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:24.387558 21885 raft_consensus.cc:493] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:24.387589 21885 raft_consensus.cc:3060] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.388106 21885 raft_consensus.cc:515] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.388227 21885 leader_election.cc:290] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [CANDIDATE]: Term 1 election: Requested vote from peers d71232d8e053429d9fc30887e484600e (127.15.205.2:40967), bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845)
I20251028 21:12:24.388358 21683 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5432e30e29a245dda2ab91568fdfef56" candidate_uuid: "5989efab16fb4953a4c614f2a38f35f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf50c9d915ce4fa8b5f425bc1a260257"
I20251028 21:12:24.388346 21759 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5432e30e29a245dda2ab91568fdfef56" candidate_uuid: "5989efab16fb4953a4c614f2a38f35f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71232d8e053429d9fc30887e484600e"
I20251028 21:12:24.388415 21683 raft_consensus.cc:3060] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.388439 21759 raft_consensus.cc:3060] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.388875 21683 raft_consensus.cc:2468] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5989efab16fb4953a4c614f2a38f35f3 in term 1.
I20251028 21:12:24.388917 21759 raft_consensus.cc:2468] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5989efab16fb4953a4c614f2a38f35f3 in term 1.
I20251028 21:12:24.389026 21798 leader_election.cc:304] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5989efab16fb4953a4c614f2a38f35f3, bf50c9d915ce4fa8b5f425bc1a260257; no voters:
I20251028 21:12:24.389115 21885 raft_consensus.cc:2804] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:24.389236 21885 raft_consensus.cc:697] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5989efab16fb4953a4c614f2a38f35f3, State: Running, Role: LEADER
I20251028 21:12:24.389303 21885 consensus_queue.cc:237] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.389432 21904 raft_consensus.cc:493] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:24.389475 21904 raft_consensus.cc:515] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.389693 21579 catalog_manager.cc:5649] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "5989efab16fb4953a4c614f2a38f35f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:24.389783 21683 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "760984c8fffe45738de29b60bf9a1a4e" candidate_uuid: "5989efab16fb4953a4c614f2a38f35f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" is_pre_election: true
I20251028 21:12:24.389878 21683 raft_consensus.cc:2468] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5989efab16fb4953a4c614f2a38f35f3 in term 0.
I20251028 21:12:24.389961 21904 leader_election.cc:290] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845), d71232d8e053429d9fc30887e484600e (127.15.205.2:40967)
I20251028 21:12:24.390020 21798 leader_election.cc:304] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5989efab16fb4953a4c614f2a38f35f3, bf50c9d915ce4fa8b5f425bc1a260257; no voters:
I20251028 21:12:24.390094 21904 raft_consensus.cc:2804] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:24.390154 21904 raft_consensus.cc:493] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:24.390079 21759 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "760984c8fffe45738de29b60bf9a1a4e" candidate_uuid: "5989efab16fb4953a4c614f2a38f35f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71232d8e053429d9fc30887e484600e" is_pre_election: true
I20251028 21:12:24.390182 21904 raft_consensus.cc:3060] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.390194 21759 raft_consensus.cc:2468] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5989efab16fb4953a4c614f2a38f35f3 in term 0.
I20251028 21:12:24.390718 21904 raft_consensus.cc:515] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.390834 21904 leader_election.cc:290] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [CANDIDATE]: Term 1 election: Requested vote from peers bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845), d71232d8e053429d9fc30887e484600e (127.15.205.2:40967)
I20251028 21:12:24.390929 21683 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "760984c8fffe45738de29b60bf9a1a4e" candidate_uuid: "5989efab16fb4953a4c614f2a38f35f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf50c9d915ce4fa8b5f425bc1a260257"
I20251028 21:12:24.390961 21759 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "760984c8fffe45738de29b60bf9a1a4e" candidate_uuid: "5989efab16fb4953a4c614f2a38f35f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71232d8e053429d9fc30887e484600e"
I20251028 21:12:24.390993 21683 raft_consensus.cc:3060] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.391005 21759 raft_consensus.cc:3060] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.391549 21759 raft_consensus.cc:2468] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5989efab16fb4953a4c614f2a38f35f3 in term 1.
I20251028 21:12:24.391600 21683 raft_consensus.cc:2468] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5989efab16fb4953a4c614f2a38f35f3 in term 1.
I20251028 21:12:24.391718 21799 leader_election.cc:304] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5989efab16fb4953a4c614f2a38f35f3, d71232d8e053429d9fc30887e484600e; no voters:
I20251028 21:12:24.391815 21904 raft_consensus.cc:2804] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:24.391870 21904 raft_consensus.cc:697] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5989efab16fb4953a4c614f2a38f35f3, State: Running, Role: LEADER
I20251028 21:12:24.391913 21904 consensus_queue.cc:237] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.392375 21579 catalog_manager.cc:5649] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "5989efab16fb4953a4c614f2a38f35f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:24.461488 21889 raft_consensus.cc:493] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:24.461591 21889 raft_consensus.cc:515] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.461784 21889 leader_election.cc:290] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d71232d8e053429d9fc30887e484600e (127.15.205.2:40967), 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:24.464403 21759 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa0913e597d64739b6233bf273d66833" candidate_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71232d8e053429d9fc30887e484600e" is_pre_election: true
I20251028 21:12:24.464481 21759 raft_consensus.cc:2468] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf50c9d915ce4fa8b5f425bc1a260257 in term 0.
I20251028 21:12:24.464613 21834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa0913e597d64739b6233bf273d66833" candidate_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5989efab16fb4953a4c614f2a38f35f3" is_pre_election: true
I20251028 21:12:24.464617 21648 leader_election.cc:304] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf50c9d915ce4fa8b5f425bc1a260257, d71232d8e053429d9fc30887e484600e; no voters:
I20251028 21:12:24.464671 21834 raft_consensus.cc:2468] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf50c9d915ce4fa8b5f425bc1a260257 in term 0.
I20251028 21:12:24.464728 21889 raft_consensus.cc:2804] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:24.464773 21889 raft_consensus.cc:493] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:24.464799 21889 raft_consensus.cc:3060] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.465435 21889 raft_consensus.cc:515] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.465577 21889 leader_election.cc:290] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [CANDIDATE]: Term 1 election: Requested vote from peers d71232d8e053429d9fc30887e484600e (127.15.205.2:40967), 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:24.465737 21759 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa0913e597d64739b6233bf273d66833" candidate_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d71232d8e053429d9fc30887e484600e"
I20251028 21:12:24.465765 21834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa0913e597d64739b6233bf273d66833" candidate_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5989efab16fb4953a4c614f2a38f35f3"
I20251028 21:12:24.465824 21759 raft_consensus.cc:3060] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.465834 21834 raft_consensus.cc:3060] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:24.466501 21759 raft_consensus.cc:2468] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf50c9d915ce4fa8b5f425bc1a260257 in term 1.
I20251028 21:12:24.466501 21834 raft_consensus.cc:2468] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf50c9d915ce4fa8b5f425bc1a260257 in term 1.
I20251028 21:12:24.466682 21648 leader_election.cc:304] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf50c9d915ce4fa8b5f425bc1a260257, d71232d8e053429d9fc30887e484600e; no voters:
I20251028 21:12:24.466784 21889 raft_consensus.cc:2804] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:24.466943 21889 raft_consensus.cc:697] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 LEADER]: Becoming Leader. State: Replica: bf50c9d915ce4fa8b5f425bc1a260257, State: Running, Role: LEADER
I20251028 21:12:24.467007 21889 consensus_queue.cc:237] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:24.467610 21579 catalog_manager.cc:5649] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1). New cstate: current_term: 1 leader_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:24.475661 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:12:24.478274 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:24.478433 16180 tablet_replica.cc:333] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257: stopping tablet replica
I20251028 21:12:24.478502 16180 raft_consensus.cc:2243] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:24.478536 16180 raft_consensus.cc:2272] T 760984c8fffe45738de29b60bf9a1a4e P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:24.478902 16180 tablet_replica.cc:333] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257: stopping tablet replica
I20251028 21:12:24.478940 16180 raft_consensus.cc:2243] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:24.478961 16180 raft_consensus.cc:2272] T 449445589fd747a4bd82016e9e3c7882 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:24.479120 16180 tablet_replica.cc:333] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257: stopping tablet replica
I20251028 21:12:24.479166 16180 raft_consensus.cc:2243] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:24.479192 16180 raft_consensus.cc:2272] T 5432e30e29a245dda2ab91568fdfef56 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:24.479362 16180 tablet_replica.cc:333] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257: stopping tablet replica
I20251028 21:12:24.479399 16180 raft_consensus.cc:2243] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:24.479423 16180 pending_rounds.cc:70] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257: Trying to abort 1 pending ops.
I20251028 21:12:24.479439 16180 pending_rounds.cc:77] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865628537335808 op_type: NO_OP noop_request { }
I20251028 21:12:24.479465 16180 raft_consensus.cc:2889] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:24.479483 16180 raft_consensus.cc:2272] T fa0913e597d64739b6233bf273d66833 P bf50c9d915ce4fa8b5f425bc1a260257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:24.481106 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
W20251028 21:12:24.812044 21723 consensus_peers.cc:597] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251028 21:12:24.821831 21798 consensus_peers.cc:597] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:24.861724 21904 consensus_queue.cc:1048] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:24.948870 21904 consensus_queue.cc:1048] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 21:12:24.950438 21798 consensus_peers.cc:597] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:24.991854 21890 consensus_queue.cc:1048] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:25.441617 21917 consensus_queue.cc:579] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Leader has been unable to successfully communicate with peer bf50c9d915ce4fa8b5f425bc1a260257 for more than 1 seconds (1.052s)
W20251028 21:12:25.442234 21580 catalog_manager.cc:5327] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 5432e30e29a245dda2ab91568fdfef56 with cas_config_opid_index -1: no extra replica candidate found for tablet 5432e30e29a245dda2ab91568fdfef56 (table test-workload [id=c768b61fbd064446960633b581fd1bb4]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (3) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20251028 21:12:25.482259 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:25.483551 21925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:25.483688 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:25.483628 21922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:25.483592 21923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:25.483899 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:25.483933 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:25.483954 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685945483953 us; error 0 us; skew 500 ppm
I20251028 21:12:25.484549 16180 webserver.cc:492] Webserver started at http://127.15.205.4:35695/ using document root <none> and password file <none>
I20251028 21:12:25.484627 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:25.484675 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:25.484746 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:25.484997 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-3-root/instance:
uuid: "0a09e846b82c4f758d6dedaeab94399e"
format_stamp: "Formatted at 2025-10-28 21:12:25 on dist-test-slave-zmkx"
I20251028 21:12:25.485849 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:25.486359 21930 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:25.486466 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:25.486514 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-3-root
uuid: "0a09e846b82c4f758d6dedaeab94399e"
format_stamp: "Formatted at 2025-10-28 21:12:25 on dist-test-slave-zmkx"
I20251028 21:12:25.486558 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1761685792944951-16180-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:25.491114 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:25.491269 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:25.491452 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:25.491619 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:25.491643 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:25.491672 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:25.491695 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:25.494738 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.4:33269
I20251028 21:12:25.495329 21993 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.4:33269 every 8 connection(s)
I20251028 21:12:25.498082 21994 heartbeater.cc:344] Connected to a master server at 127.15.205.62:34285
I20251028 21:12:25.498136 21994 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:25.498224 21994 heartbeater.cc:507] Master 127.15.205.62:34285 requested a full tablet report, sending...
I20251028 21:12:25.498430 21580 ts_manager.cc:194] Registered new tserver with Master: 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4:33269)
I20251028 21:12:25.498728 21580 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33100
I20251028 21:12:25.503809 21834 consensus_queue.cc:237] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:25.504244 21759 raft_consensus.cc:1275] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:25.504417 21917 consensus_queue.cc:1048] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251028 21:12:25.504576 21798 consensus_peers.cc:597] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:25.505031 21904 raft_consensus.cc:2955] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:25.505142 21759 raft_consensus.cc:2955] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:25.506075 21580 catalog_manager.cc:5649] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "5989efab16fb4953a4c614f2a38f35f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:25.506151 21567 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 5432e30e29a245dda2ab91568fdfef56 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 2)
W20251028 21:12:25.507668 21800 consensus_peers.cc:597] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 -> Peer 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4:33269): Couldn't send request to peer 0a09e846b82c4f758d6dedaeab94399e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5432e30e29a245dda2ab91568fdfef56. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:25.547475 21885 consensus_queue.cc:579] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Leader has been unable to successfully communicate with peer bf50c9d915ce4fa8b5f425bc1a260257 for more than 1 seconds (1.155s)
I20251028 21:12:25.548142 21834 consensus_queue.cc:237] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:25.548575 21759 raft_consensus.cc:1275] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251028 21:12:25.548677 21800 consensus_peers.cc:597] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 -> Peer 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4:33269): Couldn't send request to peer 0a09e846b82c4f758d6dedaeab94399e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 760984c8fffe45738de29b60bf9a1a4e. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:25.548754 21917 consensus_queue.cc:1048] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251028 21:12:25.548825 21798 consensus_peers.cc:597] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:25.549289 21885 raft_consensus.cc:2955] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:25.549397 21759 raft_consensus.cc:2955] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:25.550251 21579 catalog_manager.cc:5649] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e reported cstate change: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "5989efab16fb4953a4c614f2a38f35f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:25.550307 21567 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 760984c8fffe45738de29b60bf9a1a4e with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251028 21:12:25.753741 21890 consensus_queue.cc:579] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Leader has been unable to successfully communicate with peer bf50c9d915ce4fa8b5f425bc1a260257 for more than 1 seconds (1.371s)
I20251028 21:12:25.754696 21759 consensus_queue.cc:237] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:25.755311 21834 raft_consensus.cc:1275] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:25.755514 21890 consensus_queue.cc:1048] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251028 21:12:25.755740 21723 consensus_peers.cc:597] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:25.756101 22006 raft_consensus.cc:2955] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:25.756194 21834 raft_consensus.cc:2955] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:25.757136 21566 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 449445589fd747a4bd82016e9e3c7882 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251028 21:12:25.757335 21580 catalog_manager.cc:5649] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e reported cstate change: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "d71232d8e053429d9fc30887e484600e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251028 21:12:25.758194 21725 consensus_peers.cc:597] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e -> Peer 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4:33269): Couldn't send request to peer 0a09e846b82c4f758d6dedaeab94399e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 449445589fd747a4bd82016e9e3c7882. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:25.909875 22008 ts_tablet_manager.cc:933] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: Initiating tablet copy from peer 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:25.910079 22008 tablet_copy_client.cc:323] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:44093
I20251028 21:12:25.912310 21844 tablet_copy_service.cc:140] P 5989efab16fb4953a4c614f2a38f35f3: Received BeginTabletCopySession request for tablet 5432e30e29a245dda2ab91568fdfef56 from peer 0a09e846b82c4f758d6dedaeab94399e ({username='slave'} at 127.0.0.1:35988)
I20251028 21:12:25.912366 21844 tablet_copy_service.cc:161] P 5989efab16fb4953a4c614f2a38f35f3: Beginning new tablet copy session on tablet 5432e30e29a245dda2ab91568fdfef56 from peer 0a09e846b82c4f758d6dedaeab94399e at {username='slave'} at 127.0.0.1:35988: session id = 0a09e846b82c4f758d6dedaeab94399e-5432e30e29a245dda2ab91568fdfef56
I20251028 21:12:25.912824 21844 tablet_copy_source_session.cc:215] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:25.913117 22008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5432e30e29a245dda2ab91568fdfef56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:25.914017 22008 tablet_copy_client.cc:806] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:25.914109 22008 tablet_copy_client.cc:670] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:25.914590 22008 tablet_copy_client.cc:538] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:25.915094 22008 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap starting.
I20251028 21:12:25.916165 22008 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:25.916298 22008 tablet_bootstrap.cc:492] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap complete.
I20251028 21:12:25.916365 22008 ts_tablet_manager.cc:1403] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:25.916464 22008 raft_consensus.cc:359] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:25.916517 22008 raft_consensus.cc:740] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0a09e846b82c4f758d6dedaeab94399e, State: Initialized, Role: LEARNER
I20251028 21:12:25.916538 22008 consensus_queue.cc:260] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:25.916643 22008 ts_tablet_manager.cc:1434] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:25.916709 21994 heartbeater.cc:499] Master 127.15.205.62:34285 was elected leader, sending a full tablet report...
I20251028 21:12:25.916836 21844 tablet_copy_service.cc:342] P 5989efab16fb4953a4c614f2a38f35f3: Request end of tablet copy session 0a09e846b82c4f758d6dedaeab94399e-5432e30e29a245dda2ab91568fdfef56 received from {username='slave'} at 127.0.0.1:35988
I20251028 21:12:25.916877 21844 tablet_copy_service.cc:434] P 5989efab16fb4953a4c614f2a38f35f3: ending tablet copy session 0a09e846b82c4f758d6dedaeab94399e-5432e30e29a245dda2ab91568fdfef56 on tablet 5432e30e29a245dda2ab91568fdfef56 with peer 0a09e846b82c4f758d6dedaeab94399e
I20251028 21:12:25.966174 22008 ts_tablet_manager.cc:933] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: Initiating tablet copy from peer 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:25.966338 22008 tablet_copy_client.cc:323] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:44093
I20251028 21:12:25.966538 21844 tablet_copy_service.cc:140] P 5989efab16fb4953a4c614f2a38f35f3: Received BeginTabletCopySession request for tablet 760984c8fffe45738de29b60bf9a1a4e from peer 0a09e846b82c4f758d6dedaeab94399e ({username='slave'} at 127.0.0.1:35988)
I20251028 21:12:25.966580 21844 tablet_copy_service.cc:161] P 5989efab16fb4953a4c614f2a38f35f3: Beginning new tablet copy session on tablet 760984c8fffe45738de29b60bf9a1a4e from peer 0a09e846b82c4f758d6dedaeab94399e at {username='slave'} at 127.0.0.1:35988: session id = 0a09e846b82c4f758d6dedaeab94399e-760984c8fffe45738de29b60bf9a1a4e
I20251028 21:12:25.967036 21844 tablet_copy_source_session.cc:215] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:25.967375 22008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 760984c8fffe45738de29b60bf9a1a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:25.968209 22008 tablet_copy_client.cc:806] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:25.968324 22008 tablet_copy_client.cc:670] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:25.968802 22008 tablet_copy_client.cc:538] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:25.969338 22008 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap starting.
I20251028 21:12:25.970259 22008 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:25.970382 22008 tablet_bootstrap.cc:492] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap complete.
I20251028 21:12:25.970443 22008 ts_tablet_manager.cc:1403] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:25.970525 22008 raft_consensus.cc:359] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:25.970573 22008 raft_consensus.cc:740] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0a09e846b82c4f758d6dedaeab94399e, State: Initialized, Role: LEARNER
I20251028 21:12:25.970592 22008 consensus_queue.cc:260] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:25.970660 22008 ts_tablet_manager.cc:1434] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:25.970834 21844 tablet_copy_service.cc:342] P 5989efab16fb4953a4c614f2a38f35f3: Request end of tablet copy session 0a09e846b82c4f758d6dedaeab94399e-760984c8fffe45738de29b60bf9a1a4e received from {username='slave'} at 127.0.0.1:35988
I20251028 21:12:25.970868 21844 tablet_copy_service.cc:434] P 5989efab16fb4953a4c614f2a38f35f3: ending tablet copy session 0a09e846b82c4f758d6dedaeab94399e-760984c8fffe45738de29b60bf9a1a4e on tablet 760984c8fffe45738de29b60bf9a1a4e with peer 0a09e846b82c4f758d6dedaeab94399e
I20251028 21:12:25.986002 21890 raft_consensus.cc:493] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:25.986064 21890 raft_consensus.cc:515] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:25.986176 21890 leader_election.cc:290] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845), 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:25.986333 21834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa0913e597d64739b6233bf273d66833" candidate_uuid: "d71232d8e053429d9fc30887e484600e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5989efab16fb4953a4c614f2a38f35f3" is_pre_election: true
I20251028 21:12:25.986423 21834 raft_consensus.cc:2468] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d71232d8e053429d9fc30887e484600e in term 1.
I20251028 21:12:25.986577 21725 leader_election.cc:304] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5989efab16fb4953a4c614f2a38f35f3, d71232d8e053429d9fc30887e484600e; no voters:
W20251028 21:12:25.986658 21723 leader_election.cc:336] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111)
I20251028 21:12:25.986660 21890 raft_consensus.cc:2804] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 21:12:25.986743 21890 raft_consensus.cc:493] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:25.986770 21890 raft_consensus.cc:3060] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:25.987335 21890 raft_consensus.cc:515] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:25.987442 21890 leader_election.cc:290] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 2 election: Requested vote from peers bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845), 5989efab16fb4953a4c614f2a38f35f3 (127.15.205.3:44093)
I20251028 21:12:25.987555 21834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fa0913e597d64739b6233bf273d66833" candidate_uuid: "d71232d8e053429d9fc30887e484600e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5989efab16fb4953a4c614f2a38f35f3"
I20251028 21:12:25.987607 21834 raft_consensus.cc:3060] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Advancing to term 2
W20251028 21:12:25.987779 21723 leader_election.cc:336] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111)
I20251028 21:12:25.988145 21834 raft_consensus.cc:2468] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d71232d8e053429d9fc30887e484600e in term 2.
I20251028 21:12:25.988293 21725 leader_election.cc:304] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5989efab16fb4953a4c614f2a38f35f3, d71232d8e053429d9fc30887e484600e; no voters: bf50c9d915ce4fa8b5f425bc1a260257
I20251028 21:12:25.988390 21890 raft_consensus.cc:2804] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:12:25.988449 21890 raft_consensus.cc:697] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 2 LEADER]: Becoming Leader. State: Replica: d71232d8e053429d9fc30887e484600e, State: Running, Role: LEADER
I20251028 21:12:25.988510 21890 consensus_queue.cc:237] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } }
I20251028 21:12:25.988945 21579 catalog_manager.cc:5649] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e reported cstate change: term changed from 1 to 2, leader changed from bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) to d71232d8e053429d9fc30887e484600e (127.15.205.2). New cstate: current_term: 2 leader_uuid: "d71232d8e053429d9fc30887e484600e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:26.182291 22008 ts_tablet_manager.cc:933] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: Initiating tablet copy from peer d71232d8e053429d9fc30887e484600e (127.15.205.2:40967)
I20251028 21:12:26.182487 22008 tablet_copy_client.cc:323] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:40967
I20251028 21:12:26.184882 21769 tablet_copy_service.cc:140] P d71232d8e053429d9fc30887e484600e: Received BeginTabletCopySession request for tablet 449445589fd747a4bd82016e9e3c7882 from peer 0a09e846b82c4f758d6dedaeab94399e ({username='slave'} at 127.0.0.1:59452)
I20251028 21:12:26.184940 21769 tablet_copy_service.cc:161] P d71232d8e053429d9fc30887e484600e: Beginning new tablet copy session on tablet 449445589fd747a4bd82016e9e3c7882 from peer 0a09e846b82c4f758d6dedaeab94399e at {username='slave'} at 127.0.0.1:59452: session id = 0a09e846b82c4f758d6dedaeab94399e-449445589fd747a4bd82016e9e3c7882
I20251028 21:12:26.185389 21769 tablet_copy_source_session.cc:215] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:26.185689 22008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 449445589fd747a4bd82016e9e3c7882. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:26.186573 22008 tablet_copy_client.cc:806] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:26.186690 22008 tablet_copy_client.cc:670] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:26.187209 22008 tablet_copy_client.cc:538] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:26.187745 22008 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap starting.
W20251028 21:12:26.188156 21627 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver bf50c9d915ce4fa8b5f425bc1a260257 not available for placement
I20251028 21:12:26.188690 22008 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:26.188815 22008 tablet_bootstrap.cc:492] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap complete.
I20251028 21:12:26.188881 22008 ts_tablet_manager.cc:1403] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:26.188988 22008 raft_consensus.cc:359] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:26.189040 22008 raft_consensus.cc:740] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0a09e846b82c4f758d6dedaeab94399e, State: Initialized, Role: LEARNER
I20251028 21:12:26.189059 22008 consensus_queue.cc:260] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:26.189127 22008 ts_tablet_manager.cc:1434] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:26.189303 21769 tablet_copy_service.cc:342] P d71232d8e053429d9fc30887e484600e: Request end of tablet copy session 0a09e846b82c4f758d6dedaeab94399e-449445589fd747a4bd82016e9e3c7882 received from {username='slave'} at 127.0.0.1:59452
I20251028 21:12:26.189348 21769 tablet_copy_service.cc:434] P d71232d8e053429d9fc30887e484600e: ending tablet copy session 0a09e846b82c4f758d6dedaeab94399e-449445589fd747a4bd82016e9e3c7882 on tablet 449445589fd747a4bd82016e9e3c7882 with peer 0a09e846b82c4f758d6dedaeab94399e
I20251028 21:12:26.363212 21968 raft_consensus.cc:1217] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20251028 21:12:26.432459 21723 consensus_peers.cc:597] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251028 21:12:26.476493 21798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111) [suppressed 20 similar messages]
I20251028 21:12:26.522317 21968 raft_consensus.cc:1217] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251028 21:12:26.597852 21834 raft_consensus.cc:1275] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251028 21:12:26.598121 21890 consensus_queue.cc:1048] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:26.623677 21968 raft_consensus.cc:1217] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251028 21:12:26.809604 22024 consensus_queue.cc:579] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Leader has been unable to successfully communicate with peer bf50c9d915ce4fa8b5f425bc1a260257 for more than 1 seconds (1.055s)
I20251028 21:12:26.949278 21904 consensus_queue.cc:579] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Leader has been unable to successfully communicate with peer bf50c9d915ce4fa8b5f425bc1a260257 for more than 1 seconds (1.401s)
I20251028 21:12:26.952112 22019 consensus_queue.cc:579] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Leader has been unable to successfully communicate with peer bf50c9d915ce4fa8b5f425bc1a260257 for more than 1 seconds (1.448s)
I20251028 21:12:26.954679 22025 raft_consensus.cc:1064] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3: attempting to promote NON_VOTER 0a09e846b82c4f758d6dedaeab94399e to VOTER
I20251028 21:12:26.954830 22025 consensus_queue.cc:237] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } }
I20251028 21:12:26.955312 21759 raft_consensus.cc:1275] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:26.955322 21968 raft_consensus.cc:1275] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:26.955595 22027 consensus_queue.cc:1048] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:26.955695 21904 consensus_queue.cc:1048] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251028 21:12:26.955791 21798 consensus_peers.cc:597] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:26.956354 22027 raft_consensus.cc:2955] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.956480 21759 raft_consensus.cc:2955] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.957476 21579 catalog_manager.cc:5649] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e reported cstate change: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5989efab16fb4953a4c614f2a38f35f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.957506 21968 raft_consensus.cc:2955] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.959025 21834 consensus_queue.cc:237] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } }
I20251028 21:12:26.959340 21968 raft_consensus.cc:1275] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:26.959461 21759 raft_consensus.cc:1275] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:26.959537 21904 consensus_queue.cc:1048] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:26.959730 21904 consensus_queue.cc:1048] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:26.960006 22025 raft_consensus.cc:2955] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.960100 21968 raft_consensus.cc:2955] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.960690 21567 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 5432e30e29a245dda2ab91568fdfef56 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:26.960856 21578 catalog_manager.cc:5649] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 reported cstate change: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New cstate: current_term: 1 leader_uuid: "5989efab16fb4953a4c614f2a38f35f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:12:26.961092 21759 raft_consensus.cc:2955] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
W20251028 21:12:26.961974 21565 catalog_manager.cc:4712] TS bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5432e30e29a245dda2ab91568fdfef56: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111)
I20251028 21:12:26.987475 22025 raft_consensus.cc:1064] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3: attempting to promote NON_VOTER 0a09e846b82c4f758d6dedaeab94399e to VOTER
I20251028 21:12:26.987586 22025 consensus_queue.cc:237] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } }
I20251028 21:12:26.987982 21759 raft_consensus.cc:1275] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:26.988000 21968 raft_consensus.cc:1275] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:26.988157 22019 consensus_queue.cc:1048] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:26.988251 21904 consensus_queue.cc:1048] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251028 21:12:26.988291 21798 consensus_peers.cc:597] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:26.988927 21904 raft_consensus.cc:2955] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.989053 21759 raft_consensus.cc:2955] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.989886 21968 raft_consensus.cc:2955] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.989948 21578 catalog_manager.cc:5649] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e reported cstate change: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5989efab16fb4953a4c614f2a38f35f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.990718 21834 consensus_queue.cc:237] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } }
I20251028 21:12:26.991065 21759 raft_consensus.cc:1275] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:26.991065 21968 raft_consensus.cc:1275] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Refusing update from remote peer 5989efab16fb4953a4c614f2a38f35f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:26.991246 22019 consensus_queue.cc:1048] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:26.991333 22025 consensus_queue.cc:1048] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:26.991818 22027 raft_consensus.cc:2955] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.991986 21968 raft_consensus.cc:2955] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.992684 21567 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 760984c8fffe45738de29b60bf9a1a4e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:26.992753 21759 raft_consensus.cc:2955] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:26.992799 21580 catalog_manager.cc:5649] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e reported cstate change: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New cstate: current_term: 1 leader_uuid: "5989efab16fb4953a4c614f2a38f35f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:27.132479 22024 raft_consensus.cc:1064] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e: attempting to promote NON_VOTER 0a09e846b82c4f758d6dedaeab94399e to VOTER
I20251028 21:12:27.132642 22024 consensus_queue.cc:237] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } }
I20251028 21:12:27.133059 21968 raft_consensus.cc:1275] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 LEARNER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:27.133070 21834 raft_consensus.cc:1275] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:27.133291 22039 consensus_queue.cc:1048] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251028 21:12:27.133404 21723 consensus_peers.cc:597] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:27.133411 22024 consensus_queue.cc:1048] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:27.134097 22024 raft_consensus.cc:2955] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:27.134198 21968 raft_consensus.cc:2955] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:27.134968 21834 raft_consensus.cc:2955] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:27.135051 21580 catalog_manager.cc:5649] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e reported cstate change: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d71232d8e053429d9fc30887e484600e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:27.135962 21759 consensus_queue.cc:237] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } }
I20251028 21:12:27.136274 21968 raft_consensus.cc:1275] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:27.136274 21834 raft_consensus.cc:1275] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:27.136478 22021 consensus_queue.cc:1048] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:27.136550 22039 consensus_queue.cc:1048] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:27.137024 22021 raft_consensus.cc:2955] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:27.137180 21834 raft_consensus.cc:2955] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:27.137190 21968 raft_consensus.cc:2955] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:27.138008 21580 catalog_manager.cc:5649] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e reported cstate change: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New cstate: current_term: 1 leader_uuid: "d71232d8e053429d9fc30887e484600e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:12:27.138118 21566 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 449445589fd747a4bd82016e9e3c7882 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20251028 21:12:27.188344 21627 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver bf50c9d915ce4fa8b5f425bc1a260257 not available for placement
I20251028 21:12:27.404342 22024 consensus_queue.cc:579] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Leader has been unable to successfully communicate with peer bf50c9d915ce4fa8b5f425bc1a260257 for more than 1 seconds (1.416s)
I20251028 21:12:27.405208 21759 consensus_queue.cc:237] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:27.405618 21834 raft_consensus.cc:1275] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20251028 21:12:27.405663 21725 consensus_peers.cc:597] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e -> Peer 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4:33269): Couldn't send request to peer 0a09e846b82c4f758d6dedaeab94399e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa0913e597d64739b6233bf273d66833. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:27.405845 22039 consensus_queue.cc:1048] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251028 21:12:27.405930 21723 consensus_peers.cc:597] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:27.406579 22024 raft_consensus.cc:2955] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:27.406716 21834 raft_consensus.cc:2955] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:27.407593 21566 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet fa0913e597d64739b6233bf273d66833 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251028 21:12:27.407625 21579 catalog_manager.cc:5649] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) added. New cstate: current_term: 2 leader_uuid: "d71232d8e053429d9fc30887e484600e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } } }
I20251028 21:12:27.863965 22042 ts_tablet_manager.cc:933] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: Initiating tablet copy from peer d71232d8e053429d9fc30887e484600e (127.15.205.2:40967)
I20251028 21:12:27.864145 22042 tablet_copy_client.cc:323] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:40967
I20251028 21:12:27.864368 21769 tablet_copy_service.cc:140] P d71232d8e053429d9fc30887e484600e: Received BeginTabletCopySession request for tablet fa0913e597d64739b6233bf273d66833 from peer 0a09e846b82c4f758d6dedaeab94399e ({username='slave'} at 127.0.0.1:59452)
I20251028 21:12:27.864430 21769 tablet_copy_service.cc:161] P d71232d8e053429d9fc30887e484600e: Beginning new tablet copy session on tablet fa0913e597d64739b6233bf273d66833 from peer 0a09e846b82c4f758d6dedaeab94399e at {username='slave'} at 127.0.0.1:59452: session id = 0a09e846b82c4f758d6dedaeab94399e-fa0913e597d64739b6233bf273d66833
I20251028 21:12:27.864876 21769 tablet_copy_source_session.cc:215] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:27.865211 22042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0913e597d64739b6233bf273d66833. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:27.866277 22042 tablet_copy_client.cc:806] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:27.866510 22042 tablet_copy_client.cc:670] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:27.867017 22042 tablet_copy_client.cc:538] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:27.867585 22042 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap starting.
I20251028 21:12:27.868497 22042 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:27.868645 22042 tablet_bootstrap.cc:492] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: Bootstrap complete.
I20251028 21:12:27.868723 22042 ts_tablet_manager.cc:1403] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:27.868861 22042 raft_consensus.cc:359] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:27.868918 22042 raft_consensus.cc:740] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0a09e846b82c4f758d6dedaeab94399e, State: Initialized, Role: LEARNER
I20251028 21:12:27.868942 22042 consensus_queue.cc:260] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: true } }
I20251028 21:12:27.869073 22042 ts_tablet_manager.cc:1434] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:27.869252 21769 tablet_copy_service.cc:342] P d71232d8e053429d9fc30887e484600e: Request end of tablet copy session 0a09e846b82c4f758d6dedaeab94399e-fa0913e597d64739b6233bf273d66833 received from {username='slave'} at 127.0.0.1:59452
I20251028 21:12:27.869298 21769 tablet_copy_service.cc:434] P d71232d8e053429d9fc30887e484600e: ending tablet copy session 0a09e846b82c4f758d6dedaeab94399e-fa0913e597d64739b6233bf273d66833 on tablet fa0913e597d64739b6233bf273d66833 with peer 0a09e846b82c4f758d6dedaeab94399e
W20251028 21:12:28.132700 21565 catalog_manager.cc:4712] TS bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 760984c8fffe45738de29b60bf9a1a4e: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111)
W20251028 21:12:28.188550 21627 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver bf50c9d915ce4fa8b5f425bc1a260257 not available for placement
I20251028 21:12:28.403861 21968 raft_consensus.cc:1217] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20251028 21:12:28.866881 22021 consensus_queue.cc:579] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Leader has been unable to successfully communicate with peer bf50c9d915ce4fa8b5f425bc1a260257 for more than 1 seconds (1.462s)
I20251028 21:12:28.931780 22021 raft_consensus.cc:1064] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e: attempting to promote NON_VOTER 0a09e846b82c4f758d6dedaeab94399e to VOTER
I20251028 21:12:28.931963 22021 consensus_queue.cc:237] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } }
I20251028 21:12:28.932425 21968 raft_consensus.cc:1275] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 LEARNER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251028 21:12:28.932433 21834 raft_consensus.cc:1275] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251028 21:12:28.932608 22051 consensus_queue.cc:1048] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:28.932713 22052 consensus_queue.cc:1048] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251028 21:12:28.932775 21723 consensus_peers.cc:597] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e -> Peer bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): Couldn't send request to peer bf50c9d915ce4fa8b5f425bc1a260257. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:28.933439 22052 raft_consensus.cc:2955] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:28.933564 21968 raft_consensus.cc:2955] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:28.934604 21578 catalog_manager.cc:5649] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e reported cstate change: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d71232d8e053429d9fc30887e484600e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:28.934959 21834 raft_consensus.cc:2955] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 0a09e846b82c4f758d6dedaeab94399e (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "bf50c9d915ce4fa8b5f425bc1a260257" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 42845 } } peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:28.936443 21759 consensus_queue.cc:237] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } }
I20251028 21:12:28.936736 21834 raft_consensus.cc:1275] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251028 21:12:28.936780 21968 raft_consensus.cc:1275] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 FOLLOWER]: Refusing update from remote peer d71232d8e053429d9fc30887e484600e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251028 21:12:28.936897 22052 consensus_queue.cc:1048] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:28.936962 22051 consensus_queue.cc:1048] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:28.937438 22051 raft_consensus.cc:2955] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:28.937548 21834 raft_consensus.cc:2955] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:28.938169 21968 raft_consensus.cc:2955] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:28.938303 21580 catalog_manager.cc:5649] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 reported cstate change: config changed from index 3 to 4, VOTER bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1) evicted. New cstate: current_term: 2 leader_uuid: "d71232d8e053429d9fc30887e484600e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d71232d8e053429d9fc30887e484600e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 40967 } } peers { permanent_uuid: "5989efab16fb4953a4c614f2a38f35f3" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 44093 } } peers { permanent_uuid: "0a09e846b82c4f758d6dedaeab94399e" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 33269 } attrs { promote: false } } }
I20251028 21:12:28.938516 21566 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet fa0913e597d64739b6233bf273d66833 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20251028 21:12:29.160815 21565 catalog_manager.cc:4712] TS bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet fa0913e597d64739b6233bf273d66833: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111)
W20251028 21:12:31.168514 21565 catalog_manager.cc:4712] TS bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet fa0913e597d64739b6233bf273d66833: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111)
I20251028 21:12:32.520406 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:12:32.522955 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:32.523052 16180 tablet_replica.cc:333] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e: stopping tablet replica
I20251028 21:12:32.523111 16180 raft_consensus.cc:2243] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:32.523176 16180 raft_consensus.cc:2272] T fa0913e597d64739b6233bf273d66833 P d71232d8e053429d9fc30887e484600e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.523404 16180 tablet_replica.cc:333] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e: stopping tablet replica
I20251028 21:12:32.523439 16180 raft_consensus.cc:2243] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:32.523473 16180 raft_consensus.cc:2272] T 760984c8fffe45738de29b60bf9a1a4e P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.523551 16180 tablet_replica.cc:333] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e: stopping tablet replica
I20251028 21:12:32.523576 16180 raft_consensus.cc:2243] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:32.523612 16180 raft_consensus.cc:2272] T 449445589fd747a4bd82016e9e3c7882 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.523685 16180 tablet_replica.cc:333] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e: stopping tablet replica
I20251028 21:12:32.523707 16180 raft_consensus.cc:2243] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:32.523738 16180 raft_consensus.cc:2272] T 5432e30e29a245dda2ab91568fdfef56 P d71232d8e053429d9fc30887e484600e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.535444 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:12:32.536381 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:12:32.538803 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:32.538899 16180 tablet_replica.cc:333] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3: stopping tablet replica
I20251028 21:12:32.538964 16180 raft_consensus.cc:2243] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:32.539007 16180 raft_consensus.cc:2272] T 449445589fd747a4bd82016e9e3c7882 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.539106 16180 tablet_replica.cc:333] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3: stopping tablet replica
I20251028 21:12:32.539134 16180 raft_consensus.cc:2243] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:32.539166 16180 raft_consensus.cc:2272] T fa0913e597d64739b6233bf273d66833 P 5989efab16fb4953a4c614f2a38f35f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.539242 16180 tablet_replica.cc:333] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3: stopping tablet replica
I20251028 21:12:32.539321 16180 raft_consensus.cc:2243] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:32.539383 16180 raft_consensus.cc:2272] T 5432e30e29a245dda2ab91568fdfef56 P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.539464 16180 tablet_replica.cc:333] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3: stopping tablet replica
I20251028 21:12:32.539499 16180 raft_consensus.cc:2243] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:32.539525 16180 raft_consensus.cc:2272] T 760984c8fffe45738de29b60bf9a1a4e P 5989efab16fb4953a4c614f2a38f35f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.551141 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:12:32.551939 16180 tablet_server.cc:178] TabletServer@127.15.205.4:0 shutting down...
I20251028 21:12:32.554258 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:32.554354 16180 tablet_replica.cc:333] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e: stopping tablet replica
I20251028 21:12:32.554394 16180 raft_consensus.cc:2243] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:32.554440 16180 raft_consensus.cc:2272] T fa0913e597d64739b6233bf273d66833 P 0a09e846b82c4f758d6dedaeab94399e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.554558 16180 tablet_replica.cc:333] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e: stopping tablet replica
I20251028 21:12:32.554590 16180 raft_consensus.cc:2243] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:32.554629 16180 raft_consensus.cc:2272] T 449445589fd747a4bd82016e9e3c7882 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.554724 16180 tablet_replica.cc:333] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e: stopping tablet replica
I20251028 21:12:32.554749 16180 raft_consensus.cc:2243] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:32.554787 16180 raft_consensus.cc:2272] T 5432e30e29a245dda2ab91568fdfef56 P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.554874 16180 tablet_replica.cc:333] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e: stopping tablet replica
I20251028 21:12:32.554903 16180 raft_consensus.cc:2243] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:32.554941 16180 raft_consensus.cc:2272] T 760984c8fffe45738de29b60bf9a1a4e P 0a09e846b82c4f758d6dedaeab94399e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:32.566370 16180 tablet_server.cc:195] TabletServer@127.15.205.4:0 shutdown complete.
I20251028 21:12:32.567209 16180 master.cc:561] Master@127.15.205.62:34285 shutting down...
W20251028 21:12:33.226004 21565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111) [suppressed 48 similar messages]
W20251028 21:12:33.226387 21565 catalog_manager.cc:4712] TS bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet fa0913e597d64739b6233bf273d66833: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111)
W20251028 21:12:35.364182 21565 catalog_manager.cc:4712] TS bf50c9d915ce4fa8b5f425bc1a260257 (127.15.205.1:42845): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5432e30e29a245dda2ab91568fdfef56: Network error: Client connection negotiation failed: client connection to 127.15.205.1:42845: connect: Connection refused (error 111)
I20251028 21:12:36.160483 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:36.160614 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:36.160638 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P d06532e233524eccb24a5a7eef46d3b2: stopping tablet replica
I20251028 21:12:36.172274 16180 master.cc:583] Master@127.15.205.62:34285 shutdown complete.
[ OK ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering (13006 ms)
[ RUN ] AutoRebalancerTest.TestHandlingFailedTservers
I20251028 21:12:36.178140 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:43919
I20251028 21:12:36.178251 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:36.179639 22070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:36.179713 22071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:36.179749 22073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:36.179774 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:36.179939 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:36.179973 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:36.179994 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685956179994 us; error 0 us; skew 500 ppm
I20251028 21:12:36.180495 16180 webserver.cc:492] Webserver started at http://127.15.205.62:41207/ using document root <none> and password file <none>
I20251028 21:12:36.180567 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:36.180610 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:36.180655 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:36.180899 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "786a596dedd54a0aad3dd82e7ffe9811"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:36.181671 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:36.182078 22078 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.182200 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.182245 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "786a596dedd54a0aad3dd82e7ffe9811"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:36.182276 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:36.188330 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:36.188446 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:36.190567 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:43919
I20251028 21:12:36.194097 22130 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:43919 every 8 connection(s)
I20251028 21:12:36.194196 22131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:36.195060 22131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811: Bootstrap starting.
I20251028 21:12:36.195363 22131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:36.195770 22131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811: No bootstrap required, opened a new log
I20251028 21:12:36.195873 22131 raft_consensus.cc:359] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "786a596dedd54a0aad3dd82e7ffe9811" member_type: VOTER }
I20251028 21:12:36.195917 22131 raft_consensus.cc:385] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:36.195941 22131 raft_consensus.cc:740] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 786a596dedd54a0aad3dd82e7ffe9811, State: Initialized, Role: FOLLOWER
I20251028 21:12:36.195986 22131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "786a596dedd54a0aad3dd82e7ffe9811" member_type: VOTER }
I20251028 21:12:36.196020 22131 raft_consensus.cc:399] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:12:36.196034 22131 raft_consensus.cc:493] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:12:36.196060 22131 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:36.196475 22131 raft_consensus.cc:515] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "786a596dedd54a0aad3dd82e7ffe9811" member_type: VOTER }
I20251028 21:12:36.196543 22131 leader_election.cc:304] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 786a596dedd54a0aad3dd82e7ffe9811; no voters:
I20251028 21:12:36.196673 22131 leader_election.cc:290] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:12:36.196728 22134 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:36.196790 22131 sys_catalog.cc:565] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:12:36.196842 22134 raft_consensus.cc:697] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 1 LEADER]: Becoming Leader. State: Replica: 786a596dedd54a0aad3dd82e7ffe9811, State: Running, Role: LEADER
I20251028 21:12:36.196892 22134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "786a596dedd54a0aad3dd82e7ffe9811" member_type: VOTER }
I20251028 21:12:36.197129 22135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "786a596dedd54a0aad3dd82e7ffe9811" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "786a596dedd54a0aad3dd82e7ffe9811" member_type: VOTER } }
I20251028 21:12:36.197295 22135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:36.197156 22136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 786a596dedd54a0aad3dd82e7ffe9811. Latest consensus state: current_term: 1 leader_uuid: "786a596dedd54a0aad3dd82e7ffe9811" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "786a596dedd54a0aad3dd82e7ffe9811" member_type: VOTER } }
I20251028 21:12:36.197491 22136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:36.197516 22142 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:12:36.197713 22142 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:12:36.198129 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:12:36.198295 22142 catalog_manager.cc:1357] Generated new cluster ID: 478539e4cbca433bb1387d7f4b3be6fa
I20251028 21:12:36.198331 22142 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:12:36.220500 22142 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:12:36.220840 22142 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:12:36.224690 22142 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811: Generated new TSK 0
I20251028 21:12:36.224768 22142 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:12:36.230041 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:36.231181 22152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:36.231302 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:36.231364 22153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:36.231395 22155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:36.231555 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:36.231585 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:36.231597 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685956231598 us; error 0 us; skew 500 ppm
I20251028 21:12:36.232036 16180 webserver.cc:492] Webserver started at http://127.15.205.1:42029/ using document root <none> and password file <none>
I20251028 21:12:36.232098 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:36.232146 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:36.232203 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:36.232414 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "f4bf70d569404139a73af4bc15f4cac8"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:36.233184 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:36.233600 22160 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.233711 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.233748 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "f4bf70d569404139a73af4bc15f4cac8"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:36.233789 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:36.246790 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:36.246960 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:36.247076 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:36.247246 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:36.247298 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.247329 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:36.247354 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.249768 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:41039
I20251028 21:12:36.249792 22223 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:41039 every 8 connection(s)
I20251028 21:12:36.250058 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:36.251201 22228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:36.251309 22229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:36.251374 22231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:36.251463 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:36.251552 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:36.251581 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:36.251595 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685956251596 us; error 0 us; skew 500 ppm
I20251028 21:12:36.252133 16180 webserver.cc:492] Webserver started at http://127.15.205.2:37969/ using document root <none> and password file <none>
I20251028 21:12:36.252218 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:36.252259 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:36.252303 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:36.252528 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "db3e1355336c49e0a1de88a006287cd9"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:36.253454 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:36.253635 22224 heartbeater.cc:344] Connected to a master server at 127.15.205.62:43919
I20251028 21:12:36.253695 22224 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:36.253794 22224 heartbeater.cc:507] Master 127.15.205.62:43919 requested a full tablet report, sending...
I20251028 21:12:36.253922 22236 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.254014 22095 ts_manager.cc:194] Registered new tserver with Master: f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039)
I20251028 21:12:36.254058 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.254102 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "db3e1355336c49e0a1de88a006287cd9"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:36.254124 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:36.254513 22095 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46684
I20251028 21:12:36.260593 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:36.260716 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:36.260819 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:36.260947 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:36.260977 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.260999 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:36.261015 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.263375 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:44171
I20251028 21:12:36.263413 22299 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:44171 every 8 connection(s)
I20251028 21:12:36.263667 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:36.264640 22306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:36.264791 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:36.264993 22304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:36.264645 22303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:36.265292 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:36.265326 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:36.265344 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685956265344 us; error 0 us; skew 500 ppm
I20251028 21:12:36.265831 16180 webserver.cc:492] Webserver started at http://127.15.205.3:32975/ using document root <none> and password file <none>
I20251028 21:12:36.265935 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:36.265986 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:36.266021 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:36.266214 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:36.266477 22300 heartbeater.cc:344] Connected to a master server at 127.15.205.62:43919
I20251028 21:12:36.266546 22300 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:36.266655 22300 heartbeater.cc:507] Master 127.15.205.62:43919 requested a full tablet report, sending...
I20251028 21:12:36.266836 22095 ts_manager.cc:194] Registered new tserver with Master: db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171)
I20251028 21:12:36.267172 22095 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46690
I20251028 21:12:36.267189 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:36.267686 22311 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.267792 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.267841 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:36.267882 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:36.279398 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:36.279604 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:36.279735 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:36.279891 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:36.279922 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.279951 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:36.279975 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:36.282866 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:36905
I20251028 21:12:36.282907 22374 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:36905 every 8 connection(s)
I20251028 21:12:36.285130 22375 heartbeater.cc:344] Connected to a master server at 127.15.205.62:43919
I20251028 21:12:36.285172 22375 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:36.285233 22375 heartbeater.cc:507] Master 127.15.205.62:43919 requested a full tablet report, sending...
I20251028 21:12:36.285410 22095 ts_manager.cc:194] Registered new tserver with Master: 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:36.285712 22095 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46700
I20251028 21:12:36.286309 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003288914s
I20251028 21:12:37.254930 22224 heartbeater.cc:499] Master 127.15.205.62:43919 was elected leader, sending a full tablet report...
I20251028 21:12:37.267714 22300 heartbeater.cc:499] Master 127.15.205.62:43919 was elected leader, sending a full tablet report...
I20251028 21:12:37.286161 22375 heartbeater.cc:499] Master 127.15.205.62:43919 was elected leader, sending a full tablet report...
I20251028 21:12:37.311611 16180 test_util.cc:276] Using random seed: 2041642106
I20251028 21:12:37.316075 22095 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46710:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:12:37.316236 22095 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:12:37.321314 22338 tablet_service.cc:1505] Processing CreateTablet for tablet 3761d146ab3243c99e447333f8cd496b (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:37.321360 22339 tablet_service.cc:1505] Processing CreateTablet for tablet d46a3a73f9184bd29dc76964c72570c4 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:37.321399 22262 tablet_service.cc:1505] Processing CreateTablet for tablet e18d869e511a421d9955701ea3f4d964 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:37.321439 22338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3761d146ab3243c99e447333f8cd496b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.321470 22337 tablet_service.cc:1505] Processing CreateTablet for tablet e18d869e511a421d9955701ea3f4d964 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:37.321507 22262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d869e511a421d9955701ea3f4d964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.321568 22337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d869e511a421d9955701ea3f4d964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.321866 22188 tablet_service.cc:1505] Processing CreateTablet for tablet d46a3a73f9184bd29dc76964c72570c4 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:37.321954 22188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46a3a73f9184bd29dc76964c72570c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.322674 22395 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap starting.
I20251028 21:12:37.322690 22336 tablet_service.cc:1505] Processing CreateTablet for tablet de68b0b504164887810816a7a59a1ba0 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:37.322809 22336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de68b0b504164887810816a7a59a1ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.322929 22261 tablet_service.cc:1505] Processing CreateTablet for tablet de68b0b504164887810816a7a59a1ba0 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:37.323020 22261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de68b0b504164887810816a7a59a1ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.323096 22395 tablet_bootstrap.cc:654] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.323140 22187 tablet_service.cc:1505] Processing CreateTablet for tablet 3761d146ab3243c99e447333f8cd496b (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:37.323220 22187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3761d146ab3243c99e447333f8cd496b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.323915 22339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46a3a73f9184bd29dc76964c72570c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.324342 22186 tablet_service.cc:1505] Processing CreateTablet for tablet e18d869e511a421d9955701ea3f4d964 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:37.324390 22396 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Bootstrap starting.
I20251028 21:12:37.324436 22186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d869e511a421d9955701ea3f4d964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.321367 22264 tablet_service.cc:1505] Processing CreateTablet for tablet d46a3a73f9184bd29dc76964c72570c4 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:37.324767 22264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46a3a73f9184bd29dc76964c72570c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.324783 22396 tablet_bootstrap.cc:654] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.324352 22397 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap starting.
I20251028 21:12:37.325439 22397 tablet_bootstrap.cc:654] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.321290 22263 tablet_service.cc:1505] Processing CreateTablet for tablet 3761d146ab3243c99e447333f8cd496b (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:37.325716 22263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3761d146ab3243c99e447333f8cd496b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.325742 22185 tablet_service.cc:1505] Processing CreateTablet for tablet de68b0b504164887810816a7a59a1ba0 (DEFAULT_TABLE table=test-workload [id=9b3f6efa2edb4ace8decd0421a52901c]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:37.325831 22396 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: No bootstrap required, opened a new log
I20251028 21:12:37.325845 22185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de68b0b504164887810816a7a59a1ba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:37.325866 22396 ts_tablet_manager.cc:1403] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:37.326057 22396 raft_consensus.cc:359] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.326105 22396 raft_consensus.cc:385] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.326122 22396 raft_consensus.cc:740] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.326170 22396 consensus_queue.cc:260] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.326310 22396 ts_tablet_manager.cc:1434] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:37.326359 22396 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Bootstrap starting.
I20251028 21:12:37.326728 22396 tablet_bootstrap.cc:654] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.327247 22395 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: No bootstrap required, opened a new log
I20251028 21:12:37.327308 22395 ts_tablet_manager.cc:1403] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20251028 21:12:37.327382 22397 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: No bootstrap required, opened a new log
I20251028 21:12:37.327431 22397 ts_tablet_manager.cc:1403] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:12:37.327450 22395 raft_consensus.cc:359] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.327513 22395 raft_consensus.cc:385] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.327533 22395 raft_consensus.cc:740] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.327572 22395 consensus_queue.cc:260] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.327735 22395 ts_tablet_manager.cc:1434] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.327783 22395 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap starting.
I20251028 21:12:37.328161 22395 tablet_bootstrap.cc:654] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.327565 22397 raft_consensus.cc:359] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.328444 22397 raft_consensus.cc:385] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.328470 22397 raft_consensus.cc:740] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.328540 22397 consensus_queue.cc:260] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.328616 22395 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: No bootstrap required, opened a new log
I20251028 21:12:37.328661 22395 ts_tablet_manager.cc:1403] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:37.328682 22397 ts_tablet_manager.cc:1434] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:37.328732 22397 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Bootstrap starting.
I20251028 21:12:37.328795 22395 raft_consensus.cc:359] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.328848 22395 raft_consensus.cc:385] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.328864 22395 raft_consensus.cc:740] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.328899 22395 consensus_queue.cc:260] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.328965 22395 ts_tablet_manager.cc:1434] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.329015 22395 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap starting.
I20251028 21:12:37.329098 22397 tablet_bootstrap.cc:654] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.329399 22395 tablet_bootstrap.cc:654] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.329491 22396 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: No bootstrap required, opened a new log
I20251028 21:12:37.329528 22396 ts_tablet_manager.cc:1403] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:12:37.329629 22397 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: No bootstrap required, opened a new log
I20251028 21:12:37.329627 22396 raft_consensus.cc:359] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.329686 22396 raft_consensus.cc:385] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.329689 22397 ts_tablet_manager.cc:1403] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:37.329720 22396 raft_consensus.cc:740] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.329789 22396 consensus_queue.cc:260] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.329866 22396 ts_tablet_manager.cc:1434] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.329859 22397 raft_consensus.cc:359] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.329985 22396 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Bootstrap starting.
I20251028 21:12:37.329994 22397 raft_consensus.cc:385] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.330071 22397 raft_consensus.cc:740] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.330120 22397 consensus_queue.cc:260] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.330222 22397 ts_tablet_manager.cc:1434] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.330298 22397 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap starting.
I20251028 21:12:37.330406 22396 tablet_bootstrap.cc:654] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.330411 22395 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: No bootstrap required, opened a new log
I20251028 21:12:37.330529 22395 ts_tablet_manager.cc:1403] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:12:37.330660 22395 raft_consensus.cc:359] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.330717 22395 raft_consensus.cc:385] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.330740 22395 raft_consensus.cc:740] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.330750 22397 tablet_bootstrap.cc:654] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.330782 22395 consensus_queue.cc:260] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.330941 22395 ts_tablet_manager.cc:1434] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.331012 22396 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: No bootstrap required, opened a new log
I20251028 21:12:37.331056 22396 ts_tablet_manager.cc:1403] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:37.331178 22395 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap starting.
I20251028 21:12:37.331197 22396 raft_consensus.cc:359] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.331257 22396 raft_consensus.cc:385] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.331310 22396 raft_consensus.cc:740] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.331353 22396 consensus_queue.cc:260] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.331441 22396 ts_tablet_manager.cc:1434] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:37.331467 22397 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: No bootstrap required, opened a new log
I20251028 21:12:37.331493 22396 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Bootstrap starting.
I20251028 21:12:37.331616 22395 tablet_bootstrap.cc:654] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.331833 22396 tablet_bootstrap.cc:654] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.331501 22397 ts_tablet_manager.cc:1403] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:37.332253 22395 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: No bootstrap required, opened a new log
I20251028 21:12:37.332294 22395 ts_tablet_manager.cc:1403] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:37.332360 22396 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: No bootstrap required, opened a new log
I20251028 21:12:37.332406 22396 ts_tablet_manager.cc:1403] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:37.332417 22395 raft_consensus.cc:359] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.332477 22395 raft_consensus.cc:385] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.332360 22397 raft_consensus.cc:359] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.332551 22397 raft_consensus.cc:385] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.332521 22396 raft_consensus.cc:359] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.332530 22395 raft_consensus.cc:740] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.332578 22397 raft_consensus.cc:740] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.332583 22396 raft_consensus.cc:385] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.332633 22396 raft_consensus.cc:740] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.332629 22397 consensus_queue.cc:260] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.332695 22396 consensus_queue.cc:260] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.332763 22396 ts_tablet_manager.cc:1434] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.332623 22395 consensus_queue.cc:260] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.332733 22397 ts_tablet_manager.cc:1434] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.332854 22395 ts_tablet_manager.cc:1434] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:37.332859 22397 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap starting.
I20251028 21:12:37.333302 22397 tablet_bootstrap.cc:654] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:37.333941 22397 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: No bootstrap required, opened a new log
I20251028 21:12:37.333978 22397 ts_tablet_manager.cc:1403] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:37.334098 22397 raft_consensus.cc:359] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.334138 22397 raft_consensus.cc:385] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:37.334159 22397 raft_consensus.cc:740] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Initialized, Role: FOLLOWER
I20251028 21:12:37.334226 22397 consensus_queue.cc:260] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.334296 22397 ts_tablet_manager.cc:1434] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.378922 22409 raft_consensus.cc:493] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:37.379001 22409 raft_consensus.cc:515] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.379176 22409 leader_election.cc:290] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171), f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039)
I20251028 21:12:37.381382 22198 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:37.381441 22198 raft_consensus.cc:2468] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 0.
I20251028 21:12:37.381412 22274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9" is_pre_election: true
I20251028 21:12:37.381480 22274 raft_consensus.cc:2468] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 0.
I20251028 21:12:37.381564 22312 leader_election.cc:304] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:37.381636 22409 raft_consensus.cc:2804] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:37.381675 22409 raft_consensus.cc:493] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:37.381690 22409 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.382233 22409 raft_consensus.cc:515] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.382339 22409 leader_election.cc:290] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 1 election: Requested vote from peers db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171), f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039)
I20251028 21:12:37.382447 22274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9"
I20251028 21:12:37.382467 22198 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:37.382517 22274 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.382524 22198 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.383183 22274 raft_consensus.cc:2468] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 1.
I20251028 21:12:37.383183 22198 raft_consensus.cc:2468] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 1.
I20251028 21:12:37.383364 22315 leader_election.cc:304] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, db3e1355336c49e0a1de88a006287cd9; no voters:
I20251028 21:12:37.383471 22409 raft_consensus.cc:2804] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:37.383518 22409 raft_consensus.cc:697] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 LEADER]: Becoming Leader. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Running, Role: LEADER
I20251028 21:12:37.383561 22409 consensus_queue.cc:237] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:37.383994 22093 catalog_manager.cc:5649] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:37.446542 22401 raft_consensus.cc:493] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:37.446612 22401 raft_consensus.cc:515] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.446744 22401 leader_election.cc:290] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:37.449231 22198 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:37.449319 22198 raft_consensus.cc:2468] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 0.
I20251028 21:12:37.449409 22349 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" is_pre_election: true
I20251028 21:12:37.449450 22237 leader_election.cc:304] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db3e1355336c49e0a1de88a006287cd9, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:37.449467 22349 raft_consensus.cc:2468] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 0.
I20251028 21:12:37.449514 22401 raft_consensus.cc:2804] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:37.449556 22401 raft_consensus.cc:493] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:37.449573 22401 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.450119 22401 raft_consensus.cc:515] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.450232 22401 leader_election.cc:290] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 1 election: Requested vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:37.450364 22349 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
I20251028 21:12:37.450374 22198 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:37.450438 22349 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.450448 22198 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.451033 22198 raft_consensus.cc:2468] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 1.
I20251028 21:12:37.451033 22349 raft_consensus.cc:2468] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 1.
I20251028 21:12:37.451203 22237 leader_election.cc:304] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db3e1355336c49e0a1de88a006287cd9, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:37.451304 22401 raft_consensus.cc:2804] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:37.451404 22401 raft_consensus.cc:697] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 LEADER]: Becoming Leader. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Running, Role: LEADER
I20251028 21:12:37.451443 22401 consensus_queue.cc:237] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.451884 22093 catalog_manager.cc:5649] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to db3e1355336c49e0a1de88a006287cd9 (127.15.205.2). New cstate: current_term: 1 leader_uuid: "db3e1355336c49e0a1de88a006287cd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:37.540845 22401 raft_consensus.cc:493] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:37.540936 22401 raft_consensus.cc:515] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.541102 22401 leader_election.cc:290] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:37.541267 22198 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:37.541267 22349 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" is_pre_election: true
I20251028 21:12:37.541380 22198 raft_consensus.cc:2468] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 0.
I20251028 21:12:37.541397 22349 raft_consensus.cc:2468] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 0.
I20251028 21:12:37.541493 22237 leader_election.cc:304] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db3e1355336c49e0a1de88a006287cd9, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:37.541577 22401 raft_consensus.cc:2804] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:37.541628 22401 raft_consensus.cc:493] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:37.541651 22401 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.542241 22401 raft_consensus.cc:515] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.542353 22401 leader_election.cc:290] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 1 election: Requested vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:37.542487 22198 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:37.542536 22198 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.542495 22349 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
I20251028 21:12:37.542579 22349 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.543062 22198 raft_consensus.cc:2468] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 1.
I20251028 21:12:37.543174 22349 raft_consensus.cc:2468] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 1.
I20251028 21:12:37.543231 22237 leader_election.cc:304] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db3e1355336c49e0a1de88a006287cd9, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:37.543339 22401 raft_consensus.cc:2804] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:37.543399 22401 raft_consensus.cc:697] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 LEADER]: Becoming Leader. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Running, Role: LEADER
I20251028 21:12:37.543449 22401 consensus_queue.cc:237] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.544008 22093 catalog_manager.cc:5649] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to db3e1355336c49e0a1de88a006287cd9 (127.15.205.2). New cstate: current_term: 1 leader_uuid: "db3e1355336c49e0a1de88a006287cd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:37.691047 22409 raft_consensus.cc:493] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:37.691143 22409 raft_consensus.cc:515] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.691316 22409 leader_election.cc:290] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171)
I20251028 21:12:37.691484 22274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9" is_pre_election: true
I20251028 21:12:37.691486 22198 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:37.691588 22274 raft_consensus.cc:2468] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 0.
I20251028 21:12:37.691605 22198 raft_consensus.cc:2468] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 0.
I20251028 21:12:37.691720 22315 leader_election.cc:304] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, db3e1355336c49e0a1de88a006287cd9; no voters:
I20251028 21:12:37.691813 22409 raft_consensus.cc:2804] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:37.691859 22409 raft_consensus.cc:493] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:37.691879 22409 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.692478 22409 raft_consensus.cc:515] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.692584 22409 leader_election.cc:290] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 1 election: Requested vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171)
I20251028 21:12:37.692715 22274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9"
I20251028 21:12:37.692764 22274 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.692822 22198 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:37.692868 22198 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:37.693295 22274 raft_consensus.cc:2468] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 1.
I20251028 21:12:37.693346 22198 raft_consensus.cc:2468] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 1.
I20251028 21:12:37.693455 22315 leader_election.cc:304] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, db3e1355336c49e0a1de88a006287cd9; no voters:
I20251028 21:12:37.693521 22409 raft_consensus.cc:2804] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:37.693576 22409 raft_consensus.cc:697] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 LEADER]: Becoming Leader. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Running, Role: LEADER
I20251028 21:12:37.693629 22409 consensus_queue.cc:237] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:37.694159 22093 catalog_manager.cc:5649] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:37.759503 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:12:37.762105 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:37.762269 16180 tablet_replica.cc:333] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: stopping tablet replica
I20251028 21:12:37.762341 16180 raft_consensus.cc:2243] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:37.762385 16180 raft_consensus.cc:2272] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.762583 16180 tablet_replica.cc:333] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: stopping tablet replica
I20251028 21:12:37.762638 16180 raft_consensus.cc:2243] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:37.762681 16180 raft_consensus.cc:2272] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.762845 16180 tablet_replica.cc:333] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: stopping tablet replica
I20251028 21:12:37.762888 16180 raft_consensus.cc:2243] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:37.762918 16180 raft_consensus.cc:2272] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.763059 16180 tablet_replica.cc:333] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: stopping tablet replica
I20251028 21:12:37.763093 16180 raft_consensus.cc:2243] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:37.763123 16180 raft_consensus.cc:2272] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.774852 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
I20251028 21:12:37.775727 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:12:37.778097 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:37.778250 16180 tablet_replica.cc:333] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: stopping tablet replica
I20251028 21:12:37.778302 16180 raft_consensus.cc:2243] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:37.778354 16180 pending_rounds.cc:70] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Trying to abort 1 pending ops.
I20251028 21:12:37.778378 16180 pending_rounds.cc:77] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865682098434048 op_type: NO_OP noop_request { }
I20251028 21:12:37.778431 16180 raft_consensus.cc:2889] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:37.778462 16180 raft_consensus.cc:2272] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.778704 16180 tablet_replica.cc:333] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: stopping tablet replica
I20251028 21:12:37.778748 16180 raft_consensus.cc:2243] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:37.778781 16180 raft_consensus.cc:2272] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.778923 16180 tablet_replica.cc:333] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: stopping tablet replica
I20251028 21:12:37.778962 16180 raft_consensus.cc:2243] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:37.778997 16180 raft_consensus.cc:2272] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.779139 16180 tablet_replica.cc:333] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: stopping tablet replica
I20251028 21:12:37.779171 16180 raft_consensus.cc:2243] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:37.779208 16180 pending_rounds.cc:70] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Trying to abort 1 pending ops.
I20251028 21:12:37.779232 16180 pending_rounds.cc:77] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865681721524224 op_type: NO_OP noop_request { }
I20251028 21:12:37.779295 16180 raft_consensus.cc:2889] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:37.779338 16180 raft_consensus.cc:2272] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.791225 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
I20251028 21:12:37.792085 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:12:37.794749 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:37.794896 16180 tablet_replica.cc:333] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: stopping tablet replica
I20251028 21:12:37.794935 16180 raft_consensus.cc:2243] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:37.794975 16180 raft_consensus.cc:2272] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.795115 16180 tablet_replica.cc:333] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: stopping tablet replica
I20251028 21:12:37.795154 16180 raft_consensus.cc:2243] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:37.795204 16180 pending_rounds.cc:70] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Trying to abort 1 pending ops.
I20251028 21:12:37.795226 16180 pending_rounds.cc:77] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865681443434496 op_type: NO_OP noop_request { }
I20251028 21:12:37.795336 16180 raft_consensus.cc:2889] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:37.795364 16180 raft_consensus.cc:2272] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.795563 16180 tablet_replica.cc:333] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: stopping tablet replica
I20251028 21:12:37.795604 16180 raft_consensus.cc:2243] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:37.795644 16180 pending_rounds.cc:70] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Trying to abort 1 pending ops.
I20251028 21:12:37.795667 16180 pending_rounds.cc:77] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865682713620480 op_type: NO_OP noop_request { }
I20251028 21:12:37.795722 16180 raft_consensus.cc:2889] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:37.795750 16180 raft_consensus.cc:2272] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.795979 16180 tablet_replica.cc:333] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: stopping tablet replica
I20251028 21:12:37.796018 16180 raft_consensus.cc:2243] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:37.796053 16180 raft_consensus.cc:2272] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:37.797505 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:12:37.798506 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:37.799655 22426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:37.799738 22423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:37.799762 22424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:37.799911 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:37.800002 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:37.800038 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:37.800071 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685957800071 us; error 0 us; skew 500 ppm
I20251028 21:12:37.800582 16180 webserver.cc:492] Webserver started at http://127.15.205.4:36245/ using document root <none> and password file <none>
I20251028 21:12:37.800655 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:37.800699 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:37.800753 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:37.800992 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-3-root/instance:
uuid: "55b99ae813b64732ba32947d7d4ee9d9"
format_stamp: "Formatted at 2025-10-28 21:12:37 on dist-test-slave-zmkx"
I20251028 21:12:37.801754 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:37.802176 22431 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.802299 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.802347 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-3-root
uuid: "55b99ae813b64732ba32947d7d4ee9d9"
format_stamp: "Formatted at 2025-10-28 21:12:37 on dist-test-slave-zmkx"
I20251028 21:12:37.802381 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:37.809708 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:37.809831 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:37.809935 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:37.810060 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:37.810082 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.810104 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:37.810122 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:37.812613 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.4:35151
I20251028 21:12:37.814553 22494 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.4:35151 every 8 connection(s)
I20251028 21:12:37.817103 22495 heartbeater.cc:344] Connected to a master server at 127.15.205.62:43919
I20251028 21:12:37.817163 22495 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:37.817262 22495 heartbeater.cc:507] Master 127.15.205.62:43919 requested a full tablet report, sending...
I20251028 21:12:37.817469 22093 ts_manager.cc:194] Registered new tserver with Master: 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4:35151)
I20251028 21:12:37.817775 22093 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46712
W20251028 21:12:38.198443 22144 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:38.199043 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:38.199086 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:38.199455 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:38.199499 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:38.199824 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:38.199867 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
I20251028 21:12:38.818311 22495 heartbeater.cc:499] Master 127.15.205.62:43919 was elected leader, sending a full tablet report...
W20251028 21:12:39.200834 22144 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:39.201175 22082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20251028 21:12:39.201520 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:39.201555 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:39.201920 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:39.201948 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:39.202243 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:39.202270 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:40.203192 22144 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:40.203874 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:40.203943 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:40.204339 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:40.204385 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:40.204728 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:40.204787 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:41.205721 22144 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:41.206439 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:41.206486 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:41.206863 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:41.206907 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:41.207288 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:41.207330 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:42.208125 22144 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:42.208737 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:42.208777 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.2:44171: connect: Connection refused (error 111)
W20251028 21:12:42.209125 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:42.209163 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:42.209496 22144 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:42.209532 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.15.205.3:36905: connect: Connection refused (error 111)
W20251028 21:12:43.209793 22144 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver db3e1355336c49e0a1de88a006287cd9 not available for placement
I20251028 21:12:43.326643 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:43.327904 22517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:43.328001 22514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:43.328034 22515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:43.328121 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:43.328223 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:43.328249 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:43.328264 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685963328264 us; error 0 us; skew 500 ppm
I20251028 21:12:43.328826 16180 webserver.cc:492] Webserver started at http://127.15.205.1:42029/ using document root <none> and password file <none>
I20251028 21:12:43.328899 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:43.328941 16180 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251028 21:12:43.329493 16180 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.330046 22522 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.330178 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.330220 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "f4bf70d569404139a73af4bc15f4cac8"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:43.330277 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:43.335098 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:43.335290 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:43.335417 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:43.335662 22529 ts_tablet_manager.cc:542] Loading tablet metadata (0/4 complete)
I20251028 21:12:43.336743 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251028 21:12:43.336779 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:43.336818 16180 ts_tablet_manager.cc:600] Registering tablets (0/4 complete)
I20251028 21:12:43.337250 22529 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Bootstrap starting.
I20251028 21:12:43.337760 16180 ts_tablet_manager.cc:616] Registered 4 tablets
I20251028 21:12:43.337798 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:43.338212 22529 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:43.338383 22529 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Bootstrap complete.
I20251028 21:12:43.338488 22529 ts_tablet_manager.cc:1403] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:43.338660 22529 raft_consensus.cc:359] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.338718 22529 raft_consensus.cc:740] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.338794 22529 consensus_queue.cc:260] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.338939 22529 ts_tablet_manager.cc:1434] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:43.339006 22529 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap starting.
I20251028 21:12:43.339784 22529 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:43.339968 22529 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap complete.
I20251028 21:12:43.340060 22529 ts_tablet_manager.cc:1403] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:43.340162 22529 raft_consensus.cc:359] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.340206 22529 raft_consensus.cc:740] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.340265 22529 consensus_queue.cc:260] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.340349 22529 ts_tablet_manager.cc:1434] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.340397 22529 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap starting.
I20251028 21:12:43.341146 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:41039
I20251028 21:12:43.341154 22529 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:43.341161 22590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:41039 every 8 connection(s)
I20251028 21:12:43.341351 22529 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap complete.
W20251028 21:12:43.341390 22592 tablet.cc:2378] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251028 21:12:43.341424 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:12:43.341441 22529 ts_tablet_manager.cc:1403] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:43.341535 22529 raft_consensus.cc:359] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.341593 22529 raft_consensus.cc:740] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.341640 22529 consensus_queue.cc:260] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.341724 22529 ts_tablet_manager.cc:1434] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.341765 22529 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap starting.
I20251028 21:12:43.342545 22529 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20251028 21:12:43.342584 22598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:43.342605 22595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:43.342720 22529 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Bootstrap complete.
I20251028 21:12:43.342825 22529 ts_tablet_manager.cc:1403] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:43.342911 22529 raft_consensus.cc:359] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:43.342941 22529 raft_consensus.cc:740] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.343003 22529 consensus_queue.cc:260] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:43.343068 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:43.343098 22529 ts_tablet_manager.cc:1434] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20251028 21:12:43.343330 22596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:43.343441 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:43.343475 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:43.343490 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685963343491 us; error 0 us; skew 500 ppm
I20251028 21:12:43.344056 16180 webserver.cc:492] Webserver started at http://127.15.205.2:37969/ using document root <none> and password file <none>
I20251028 21:12:43.344134 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:43.344167 16180 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251028 21:12:43.344673 16180 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:43.345156 22604 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.345273 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.345310 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "db3e1355336c49e0a1de88a006287cd9"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:43.345358 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:43.348523 22591 heartbeater.cc:344] Connected to a master server at 127.15.205.62:43919
I20251028 21:12:43.348572 22591 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:43.348656 22591 heartbeater.cc:507] Master 127.15.205.62:43919 requested a full tablet report, sending...
I20251028 21:12:43.348909 22093 ts_manager.cc:194] Re-registered known tserver with Master: f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039)
I20251028 21:12:43.349242 22093 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46714
I20251028 21:12:43.349545 22591 heartbeater.cc:499] Master 127.15.205.62:43919 was elected leader, sending a full tablet report...
I20251028 21:12:43.358471 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:43.358632 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:43.358743 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:43.359983 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251028 21:12:43.360014 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:43.360378 22611 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Bootstrap starting.
I20251028 21:12:43.360855 16180 ts_tablet_manager.cc:616] Registered 4 tablets
I20251028 21:12:43.360888 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:43.361318 22611 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251028 21:12:43.361536 22611 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Bootstrap complete.
I20251028 21:12:43.361629 22611 ts_tablet_manager.cc:1403] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:43.361764 22611 raft_consensus.cc:359] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.361896 22611 raft_consensus.cc:740] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.361994 22611 consensus_queue.cc:260] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.362149 22611 ts_tablet_manager.cc:1434] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.362221 22611 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Bootstrap starting.
I20251028 21:12:43.363435 22611 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251028 21:12:43.363643 22611 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Bootstrap complete.
I20251028 21:12:43.363731 22611 ts_tablet_manager.cc:1403] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20251028 21:12:43.363867 22611 raft_consensus.cc:359] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.363926 22611 raft_consensus.cc:740] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.363974 22611 consensus_queue.cc:260] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.364065 22611 ts_tablet_manager.cc:1434] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.364113 22611 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Bootstrap starting.
I20251028 21:12:43.364468 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:44171
I20251028 21:12:43.364508 22671 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:44171 every 8 connection(s)
I20251028 21:12:43.364717 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:12:43.365325 22611 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:43.365548 22611 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Bootstrap complete.
I20251028 21:12:43.365635 22611 ts_tablet_manager.cc:1403] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
W20251028 21:12:43.365721 22675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:43.365746 22678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:43.365821 22611 raft_consensus.cc:359] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.365871 22611 raft_consensus.cc:740] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Initialized, Role: FOLLOWER
W20251028 21:12:43.365909 22676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:43.366026 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:43.365949 22611 consensus_queue.cc:260] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.366158 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:43.366214 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:43.366235 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685963366234 us; error 0 us; skew 500 ppm
I20251028 21:12:43.366250 22611 ts_tablet_manager.cc:1434] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:43.366304 22611 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Bootstrap starting.
I20251028 21:12:43.366740 16180 webserver.cc:492] Webserver started at http://127.15.205.3:32975/ using document root <none> and password file <none>
I20251028 21:12:43.366806 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:43.366832 16180 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251028 21:12:43.367341 22611 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:43.367400 16180 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:43.367522 22611 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Bootstrap complete.
I20251028 21:12:43.367622 22611 ts_tablet_manager.cc:1403] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:43.367772 22611 raft_consensus.cc:359] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:43.367834 22611 raft_consensus.cc:740] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.367885 22611 consensus_queue.cc:260] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:43.367981 22611 ts_tablet_manager.cc:1434] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:43.368081 22685 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.368201 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:43.368255 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
format_stamp: "Formatted at 2025-10-28 21:12:36 on dist-test-slave-zmkx"
I20251028 21:12:43.368315 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:43.371390 22672 heartbeater.cc:344] Connected to a master server at 127.15.205.62:43919
I20251028 21:12:43.371455 22672 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:43.371560 22672 heartbeater.cc:507] Master 127.15.205.62:43919 requested a full tablet report, sending...
I20251028 21:12:43.371802 22093 ts_manager.cc:194] Re-registered known tserver with Master: db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171)
I20251028 21:12:43.372207 22093 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46722
I20251028 21:12:43.372567 22672 heartbeater.cc:499] Master 127.15.205.62:43919 was elected leader, sending a full tablet report...
I20251028 21:12:43.395871 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:43.396039 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:43.396132 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:43.397346 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251028 21:12:43.397380 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:43.397737 22692 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap starting.
I20251028 21:12:43.398231 16180 ts_tablet_manager.cc:616] Registered 4 tablets
I20251028 21:12:43.398263 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:43.398775 22692 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:43.398907 22692 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap complete.
I20251028 21:12:43.398988 22692 ts_tablet_manager.cc:1403] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:43.399121 22692 raft_consensus.cc:359] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.399174 22692 raft_consensus.cc:740] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.399214 22692 consensus_queue.cc:260] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.399408 22692 ts_tablet_manager.cc:1434] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:43.399466 22692 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap starting.
I20251028 21:12:43.400439 22692 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:43.400595 22692 tablet_bootstrap.cc:492] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap complete.
I20251028 21:12:43.400683 22692 ts_tablet_manager.cc:1403] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:43.400842 22692 raft_consensus.cc:359] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.400903 22692 raft_consensus.cc:740] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.400945 22692 consensus_queue.cc:260] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.401024 22692 ts_tablet_manager.cc:1434] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:43.401069 22692 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap starting.
I20251028 21:12:43.401993 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:36905
I20251028 21:12:43.402009 22692 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251028 21:12:43.402031 22753 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:36905 every 8 connection(s)
I20251028 21:12:43.402225 22692 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap complete.
I20251028 21:12:43.402321 22692 ts_tablet_manager.cc:1403] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:43.402448 22692 raft_consensus.cc:359] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.402505 22692 raft_consensus.cc:740] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.402562 22692 consensus_queue.cc:260] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:43.402644 22692 ts_tablet_manager.cc:1434] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.402697 22692 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap starting.
I20251028 21:12:43.403718 22692 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251028 21:12:43.403884 22692 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Bootstrap complete.
I20251028 21:12:43.403975 22692 ts_tablet_manager.cc:1403] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:43.404091 22692 raft_consensus.cc:359] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:43.404145 22692 raft_consensus.cc:740] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Initialized, Role: FOLLOWER
I20251028 21:12:43.404201 22692 consensus_queue.cc:260] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:43.404279 22692 ts_tablet_manager.cc:1434] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:43.405622 22754 heartbeater.cc:344] Connected to a master server at 127.15.205.62:43919
I20251028 21:12:43.405669 22754 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:43.405747 22754 heartbeater.cc:507] Master 127.15.205.62:43919 requested a full tablet report, sending...
I20251028 21:12:43.405993 22093 ts_manager.cc:194] Re-registered known tserver with Master: 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:43.406301 22093 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46724
I20251028 21:12:43.406605 22754 heartbeater.cc:499] Master 127.15.205.62:43919 was elected leader, sending a full tablet report...
W20251028 21:12:44.213184 22144 auto_rebalancer.cc:253] failed to send replica move request: Illegal state: Replica db3e1355336c49e0a1de88a006287cd9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } }
W20251028 21:12:44.213526 22144 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 3761d146ab3243c99e447333f8cd496b, TS 0a66afdb6b2e4b0c873b732c6bb6c3a2 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
W20251028 21:12:44.213557 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 3761d146ab3243c99e447333f8cd496b, TS 0a66afdb6b2e4b0c873b732c6bb6c3a2 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
W20251028 21:12:44.216118 22144 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet d46a3a73f9184bd29dc76964c72570c4, TS f4bf70d569404139a73af4bc15f4cac8 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
W20251028 21:12:44.216163 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet d46a3a73f9184bd29dc76964c72570c4, TS f4bf70d569404139a73af4bc15f4cac8 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
W20251028 21:12:44.216497 22144 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet e18d869e511a421d9955701ea3f4d964, TS db3e1355336c49e0a1de88a006287cd9 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
W20251028 21:12:44.216538 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet e18d869e511a421d9955701ea3f4d964, TS db3e1355336c49e0a1de88a006287cd9 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.601732 22761 raft_consensus.cc:493] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:44.601827 22761 raft_consensus.cc:515] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.602039 22761 leader_election.cc:290] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:44.604925 22645 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "f4bf70d569404139a73af4bc15f4cac8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9" is_pre_election: true
I20251028 21:12:44.604975 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "f4bf70d569404139a73af4bc15f4cac8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" is_pre_election: true
I20251028 21:12:44.605031 22645 raft_consensus.cc:2410] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f4bf70d569404139a73af4bc15f4cac8 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251028 21:12:44.605046 22727 raft_consensus.cc:2468] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4bf70d569404139a73af4bc15f4cac8 in term 1.
I20251028 21:12:44.605203 22524 leader_election.cc:304] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:44.605288 22761 raft_consensus.cc:2804] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 21:12:44.605330 22761 raft_consensus.cc:493] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:44.605348 22761 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.605938 22761 raft_consensus.cc:515] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.606050 22761 leader_election.cc:290] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [CANDIDATE]: Term 2 election: Requested vote from peers db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:44.606190 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "f4bf70d569404139a73af4bc15f4cac8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
I20251028 21:12:44.606243 22727 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.606805 22727 raft_consensus.cc:2468] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4bf70d569404139a73af4bc15f4cac8 in term 2.
I20251028 21:12:44.606881 22645 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "f4bf70d569404139a73af4bc15f4cac8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9"
I20251028 21:12:44.606925 22645 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.606961 22524 leader_election.cc:304] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:44.607059 22761 raft_consensus.cc:2804] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:12:44.607197 22761 raft_consensus.cc:697] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 2 LEADER]: Becoming Leader. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Running, Role: LEADER
I20251028 21:12:44.607331 22761 consensus_queue.cc:237] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.607431 22765 raft_consensus.cc:493] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:44.607540 22765 raft_consensus.cc:515] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:44.607636 22645 raft_consensus.cc:2410] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f4bf70d569404139a73af4bc15f4cac8 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251028 21:12:44.607725 22765 leader_election.cc:290] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905), f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039)
I20251028 21:12:44.607921 22093 catalog_manager.cc:5649] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 reported cstate change: term changed from 1 to 2, leader changed from db3e1355336c49e0a1de88a006287cd9 (127.15.205.2) to f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1). New cstate: current_term: 2 leader_uuid: "f4bf70d569404139a73af4bc15f4cac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:44.610170 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" is_pre_election: true
I20251028 21:12:44.610247 22727 raft_consensus.cc:2410] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db3e1355336c49e0a1de88a006287cd9 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251028 21:12:44.610589 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:44.610667 22564 raft_consensus.cc:2468] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 1.
I20251028 21:12:44.610800 22605 leader_election.cc:304] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db3e1355336c49e0a1de88a006287cd9, f4bf70d569404139a73af4bc15f4cac8; no voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2
I20251028 21:12:44.610888 22765 raft_consensus.cc:2804] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 21:12:44.610924 22765 raft_consensus.cc:493] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:44.610937 22765 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.611421 22765 raft_consensus.cc:515] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:44.611516 22765 leader_election.cc:290] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 2 election: Requested vote from peers 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905), f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039)
I20251028 21:12:44.611645 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
I20251028 21:12:44.611701 22727 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.612105 22727 raft_consensus.cc:2410] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate db3e1355336c49e0a1de88a006287cd9 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251028 21:12:44.612212 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:44.612268 22564 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.612670 22564 raft_consensus.cc:2468] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 2.
I20251028 21:12:44.612809 22605 leader_election.cc:304] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db3e1355336c49e0a1de88a006287cd9, f4bf70d569404139a73af4bc15f4cac8; no voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2
I20251028 21:12:44.612896 22765 raft_consensus.cc:2804] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:12:44.613018 22765 raft_consensus.cc:697] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 2 LEADER]: Becoming Leader. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Running, Role: LEADER
I20251028 21:12:44.613070 22765 consensus_queue.cc:237] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:44.613523 22093 catalog_manager.cc:5649] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 reported cstate change: term changed from 1 to 2, leader changed from 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3) to db3e1355336c49e0a1de88a006287cd9 (127.15.205.2). New cstate: current_term: 2 leader_uuid: "db3e1355336c49e0a1de88a006287cd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:44.738131 22761 raft_consensus.cc:493] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:44.738240 22761 raft_consensus.cc:515] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.738409 22761 leader_election.cc:290] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:44.738618 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "f4bf70d569404139a73af4bc15f4cac8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" is_pre_election: true
I20251028 21:12:44.738685 22727 raft_consensus.cc:2468] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4bf70d569404139a73af4bc15f4cac8 in term 1.
I20251028 21:12:44.738657 22645 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "f4bf70d569404139a73af4bc15f4cac8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9" is_pre_election: true
I20251028 21:12:44.738746 22645 raft_consensus.cc:2410] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f4bf70d569404139a73af4bc15f4cac8 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251028 21:12:44.738822 22524 leader_election.cc:304] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:44.738901 22761 raft_consensus.cc:2804] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 21:12:44.738945 22761 raft_consensus.cc:493] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:44.738962 22761 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.739602 22761 raft_consensus.cc:515] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.739702 22761 leader_election.cc:290] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [CANDIDATE]: Term 2 election: Requested vote from peers db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:44.739817 22645 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "f4bf70d569404139a73af4bc15f4cac8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9"
I20251028 21:12:44.739836 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "f4bf70d569404139a73af4bc15f4cac8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
I20251028 21:12:44.739867 22645 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.739877 22727 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.740365 22645 raft_consensus.cc:2410] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f4bf70d569404139a73af4bc15f4cac8 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251028 21:12:44.740406 22727 raft_consensus.cc:2468] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4bf70d569404139a73af4bc15f4cac8 in term 2.
I20251028 21:12:44.740535 22524 leader_election.cc:304] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters: db3e1355336c49e0a1de88a006287cd9
I20251028 21:12:44.740617 22761 raft_consensus.cc:2804] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:12:44.740670 22761 raft_consensus.cc:697] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 2 LEADER]: Becoming Leader. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Running, Role: LEADER
I20251028 21:12:44.740720 22761 consensus_queue.cc:237] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.741189 22093 catalog_manager.cc:5649] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 reported cstate change: term changed from 1 to 2, leader changed from db3e1355336c49e0a1de88a006287cd9 (127.15.205.2) to f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1). New cstate: current_term: 2 leader_uuid: "f4bf70d569404139a73af4bc15f4cac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:44.927973 22765 raft_consensus.cc:493] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:44.928063 22765 raft_consensus.cc:515] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.928198 22765 leader_election.cc:290] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:44.928409 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:44.928414 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" is_pre_election: true
I20251028 21:12:44.928542 22727 raft_consensus.cc:2468] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 2.
I20251028 21:12:44.928697 22606 leader_election.cc:304] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, db3e1355336c49e0a1de88a006287cd9; no voters: f4bf70d569404139a73af4bc15f4cac8
I20251028 21:12:44.928786 22765 raft_consensus.cc:2804] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251028 21:12:44.928831 22765 raft_consensus.cc:493] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:44.928862 22765 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:44.929466 22765 raft_consensus.cc:515] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.929579 22765 leader_election.cc:290] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 3 election: Requested vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:44.929694 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:44.929724 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3761d146ab3243c99e447333f8cd496b" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
I20251028 21:12:44.929766 22727 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:44.930405 22727 raft_consensus.cc:2468] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 3.
I20251028 21:12:44.930594 22606 leader_election.cc:304] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, db3e1355336c49e0a1de88a006287cd9; no voters: f4bf70d569404139a73af4bc15f4cac8
I20251028 21:12:44.930673 22765 raft_consensus.cc:2804] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:12:44.930717 22765 raft_consensus.cc:697] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 3 LEADER]: Becoming Leader. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Running, Role: LEADER
I20251028 21:12:44.930766 22765 consensus_queue.cc:237] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.931406 22093 catalog_manager.cc:5649] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 reported cstate change: term changed from 2 to 3, leader changed from f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1) to db3e1355336c49e0a1de88a006287cd9 (127.15.205.2). New cstate: current_term: 3 leader_uuid: "db3e1355336c49e0a1de88a006287cd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:44.998168 22765 raft_consensus.cc:493] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:44.998287 22765 raft_consensus.cc:515] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.998453 22765 leader_election.cc:290] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:44.998638 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:44.998633 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" is_pre_election: true
I20251028 21:12:44.998745 22564 raft_consensus.cc:2468] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 1.
I20251028 21:12:44.998752 22727 raft_consensus.cc:2410] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db3e1355336c49e0a1de88a006287cd9 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251028 21:12:44.998881 22605 leader_election.cc:304] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db3e1355336c49e0a1de88a006287cd9, f4bf70d569404139a73af4bc15f4cac8; no voters:
I20251028 21:12:44.998991 22765 raft_consensus.cc:2804] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 21:12:44.999042 22765 raft_consensus.cc:493] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:44.999056 22765 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:44.999720 22765 raft_consensus.cc:515] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:44.999830 22765 leader_election.cc:290] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 2 election: Requested vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:44.999943 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:44.999998 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
I20251028 21:12:45.000005 22564 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:45.000064 22727 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:45.000567 22727 raft_consensus.cc:2410] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate db3e1355336c49e0a1de88a006287cd9 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251028 21:12:45.000597 22564 raft_consensus.cc:2468] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 2.
I20251028 21:12:45.000728 22605 leader_election.cc:304] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db3e1355336c49e0a1de88a006287cd9, f4bf70d569404139a73af4bc15f4cac8; no voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2
I20251028 21:12:45.000805 22765 raft_consensus.cc:2804] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:12:45.000854 22765 raft_consensus.cc:697] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 2 LEADER]: Becoming Leader. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Running, Role: LEADER
I20251028 21:12:45.000900 22765 consensus_queue.cc:237] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.001443 22093 catalog_manager.cc:5649] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 reported cstate change: term changed from 1 to 2, leader changed from 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3) to db3e1355336c49e0a1de88a006287cd9 (127.15.205.2). New cstate: current_term: 2 leader_uuid: "db3e1355336c49e0a1de88a006287cd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:45.062104 22779 raft_consensus.cc:493] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:45.062197 22779 raft_consensus.cc:515] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:45.062404 22779 leader_election.cc:290] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171), f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039)
I20251028 21:12:45.065804 22645 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9" is_pre_election: true
I20251028 21:12:45.065804 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:45.065949 22564 raft_consensus.cc:2468] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 2.
I20251028 21:12:45.066073 22686 leader_election.cc:304] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters: db3e1355336c49e0a1de88a006287cd9
I20251028 21:12:45.066182 22779 raft_consensus.cc:2804] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251028 21:12:45.066224 22779 raft_consensus.cc:493] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:45.066238 22779 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.066781 22779 raft_consensus.cc:515] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:45.066885 22779 leader_election.cc:290] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 3 election: Requested vote from peers db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171), f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039)
I20251028 21:12:45.067023 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:45.067035 22645 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d46a3a73f9184bd29dc76964c72570c4" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9"
I20251028 21:12:45.067070 22564 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.067701 22564 raft_consensus.cc:2468] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 3.
I20251028 21:12:45.067857 22686 leader_election.cc:304] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters: db3e1355336c49e0a1de88a006287cd9
I20251028 21:12:45.067946 22779 raft_consensus.cc:2804] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:12:45.068105 22779 raft_consensus.cc:697] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 LEADER]: Becoming Leader. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Running, Role: LEADER
I20251028 21:12:45.068157 22779 consensus_queue.cc:237] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:45.068619 22093 catalog_manager.cc:5649] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 reported cstate change: term changed from 2 to 3, leader changed from db3e1355336c49e0a1de88a006287cd9 (127.15.205.2) to 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3). New cstate: current_term: 3 leader_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:45.074537 22564 raft_consensus.cc:1240] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Rejecting Update request from peer db3e1355336c49e0a1de88a006287cd9 for earlier term 2. Current term is 3. Ops: []
I20251028 21:12:45.074740 22765 consensus_queue.cc:1059] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.074829 22765 raft_consensus.cc:3055] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:12:45.074863 22765 raft_consensus.cc:740] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Running, Role: LEADER
I20251028 21:12:45.074918 22765 consensus_queue.cc:260] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:45.074985 22765 raft_consensus.cc:3060] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.111806 22727 raft_consensus.cc:1240] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Rejecting Update request from peer f4bf70d569404139a73af4bc15f4cac8 for earlier term 2. Current term is 3. Ops: []
I20251028 21:12:45.112049 22761 consensus_queue.cc:1059] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.112155 22761 raft_consensus.cc:3055] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:12:45.112187 22761 raft_consensus.cc:740] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Running, Role: LEADER
I20251028 21:12:45.112239 22761 consensus_queue.cc:260] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.112309 22761 raft_consensus.cc:3060] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.114513 22779 raft_consensus.cc:493] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:45.114562 22779 raft_consensus.cc:515] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.114681 22779 leader_election.cc:290] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171)
I20251028 21:12:45.114822 22645 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9" is_pre_election: true
I20251028 21:12:45.114841 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:45.114913 22564 raft_consensus.cc:2468] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 2.
I20251028 21:12:45.115046 22686 leader_election.cc:304] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters: db3e1355336c49e0a1de88a006287cd9
I20251028 21:12:45.115129 22779 raft_consensus.cc:2804] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251028 21:12:45.115165 22779 raft_consensus.cc:493] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:45.115193 22779 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.115686 22779 raft_consensus.cc:515] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.115793 22779 leader_election.cc:290] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 3 election: Requested vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171)
I20251028 21:12:45.115901 22645 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "db3e1355336c49e0a1de88a006287cd9"
I20251028 21:12:45.115898 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e18d869e511a421d9955701ea3f4d964" candidate_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:45.115994 22564 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.116436 22564 raft_consensus.cc:2468] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a66afdb6b2e4b0c873b732c6bb6c3a2 in term 3.
I20251028 21:12:45.116544 22686 leader_election.cc:304] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, f4bf70d569404139a73af4bc15f4cac8; no voters: db3e1355336c49e0a1de88a006287cd9
I20251028 21:12:45.116614 22779 raft_consensus.cc:2804] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:12:45.116657 22779 raft_consensus.cc:697] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 LEADER]: Becoming Leader. State: Replica: 0a66afdb6b2e4b0c873b732c6bb6c3a2, State: Running, Role: LEADER
I20251028 21:12:45.116703 22779 consensus_queue.cc:237] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.117120 22093 catalog_manager.cc:5649] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 reported cstate change: term changed from 2 to 3, leader changed from db3e1355336c49e0a1de88a006287cd9 (127.15.205.2) to 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3). New cstate: current_term: 3 leader_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:45.192418 22765 raft_consensus.cc:493] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:45.192512 22765 raft_consensus.cc:515] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.192674 22765 leader_election.cc:290] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:45.192879 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8" is_pre_election: true
I20251028 21:12:45.192885 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" is_pre_election: true
I20251028 21:12:45.192992 22727 raft_consensus.cc:2468] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 2.
I20251028 21:12:45.193152 22606 leader_election.cc:304] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, db3e1355336c49e0a1de88a006287cd9; no voters: f4bf70d569404139a73af4bc15f4cac8
I20251028 21:12:45.193234 22765 raft_consensus.cc:2804] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251028 21:12:45.193275 22765 raft_consensus.cc:493] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:45.193305 22765 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.193861 22765 raft_consensus.cc:515] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.193974 22765 leader_election.cc:290] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 3 election: Requested vote from peers f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039), 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:45.194096 22564 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f4bf70d569404139a73af4bc15f4cac8"
I20251028 21:12:45.194106 22727 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de68b0b504164887810816a7a59a1ba0" candidate_uuid: "db3e1355336c49e0a1de88a006287cd9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2"
I20251028 21:12:45.194161 22727 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.194689 22727 raft_consensus.cc:2468] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e1355336c49e0a1de88a006287cd9 in term 3.
I20251028 21:12:45.194841 22606 leader_election.cc:304] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a66afdb6b2e4b0c873b732c6bb6c3a2, db3e1355336c49e0a1de88a006287cd9; no voters: f4bf70d569404139a73af4bc15f4cac8
I20251028 21:12:45.194924 22765 raft_consensus.cc:2804] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Leader election won for term 3
I20251028 21:12:45.194972 22765 raft_consensus.cc:697] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 3 LEADER]: Becoming Leader. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Running, Role: LEADER
I20251028 21:12:45.195020 22765 consensus_queue.cc:237] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.195544 22093 catalog_manager.cc:5649] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 reported cstate change: term changed from 2 to 3, leader changed from f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1) to db3e1355336c49e0a1de88a006287cd9 (127.15.205.2). New cstate: current_term: 3 leader_uuid: "db3e1355336c49e0a1de88a006287cd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: UNKNOWN } } }
W20251028 21:12:45.217311 22144 auto_rebalancer.cc:253] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251028 21:12:45.217705 22144 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet e18d869e511a421d9955701ea3f4d964, TS f4bf70d569404139a73af4bc15f4cac8 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
W20251028 21:12:45.217741 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet e18d869e511a421d9955701ea3f4d964, TS f4bf70d569404139a73af4bc15f4cac8 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
W20251028 21:12:45.218083 22144 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet de68b0b504164887810816a7a59a1ba0, TS 0a66afdb6b2e4b0c873b732c6bb6c3a2 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
W20251028 21:12:45.218114 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet de68b0b504164887810816a7a59a1ba0, TS 0a66afdb6b2e4b0c873b732c6bb6c3a2 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
W20251028 21:12:45.218396 22144 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet d46a3a73f9184bd29dc76964c72570c4, TS db3e1355336c49e0a1de88a006287cd9 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
W20251028 21:12:45.218428 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet d46a3a73f9184bd29dc76964c72570c4, TS db3e1355336c49e0a1de88a006287cd9 -> TS 55b99ae813b64732ba32947d7d4ee9d9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } }
I20251028 21:12:45.239545 22645 raft_consensus.cc:1240] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 3 LEADER]: Rejecting Update request from peer f4bf70d569404139a73af4bc15f4cac8 for earlier term 2. Current term is 3. Ops: []
I20251028 21:12:45.239735 22761 consensus_queue.cc:1059] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.239804 22761 consensus_queue.cc:1243] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [LEADER]: Peer db3e1355336c49e0a1de88a006287cd9 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251028 21:12:45.239836 22761 raft_consensus.cc:3055] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:12:45.239861 22761 raft_consensus.cc:740] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f4bf70d569404139a73af4bc15f4cac8, State: Running, Role: LEADER
I20251028 21:12:45.239910 22761 consensus_queue.cc:260] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.239990 22761 raft_consensus.cc:3060] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.322710 22727 raft_consensus.cc:1275] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Refusing update from remote peer db3e1355336c49e0a1de88a006287cd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251028 21:12:45.322937 22765 consensus_queue.cc:1048] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.324299 22564 raft_consensus.cc:1275] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Refusing update from remote peer db3e1355336c49e0a1de88a006287cd9: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251028 21:12:45.324529 22788 consensus_queue.cc:1048] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.324627 22788 consensus_queue.cc:1243] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Peer f4bf70d569404139a73af4bc15f4cac8 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251028 21:12:45.324865 22564 pending_rounds.cc:85] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Aborting all ops after (but not including) 0
I20251028 21:12:45.324904 22564 pending_rounds.cc:107] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251028 21:12:45.324930 22564 raft_consensus.cc:2889] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251028 21:12:45.446168 22564 raft_consensus.cc:1275] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Refusing update from remote peer 0a66afdb6b2e4b0c873b732c6bb6c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251028 21:12:45.446405 22779 consensus_queue.cc:1048] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.447731 22645 raft_consensus.cc:1275] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Refusing update from remote peer 0a66afdb6b2e4b0c873b732c6bb6c3a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251028 21:12:45.447940 22792 consensus_queue.cc:1048] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.448021 22792 consensus_queue.cc:1243] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Peer db3e1355336c49e0a1de88a006287cd9 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251028 21:12:45.448244 22645 pending_rounds.cc:85] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Aborting all ops after (but not including) 0
I20251028 21:12:45.448287 22645 pending_rounds.cc:107] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251028 21:12:45.448309 22645 raft_consensus.cc:2889] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251028 21:12:45.452092 22564 raft_consensus.cc:1240] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Rejecting Update request from peer db3e1355336c49e0a1de88a006287cd9 for earlier term 2. Current term is 3. Ops: []
I20251028 21:12:45.452276 22774 consensus_queue.cc:1059] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.452366 22774 raft_consensus.cc:3055] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 2 LEADER]: Stepping down as leader of term 2
I20251028 21:12:45.452399 22774 raft_consensus.cc:740] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db3e1355336c49e0a1de88a006287cd9, State: Running, Role: LEADER
I20251028 21:12:45.452456 22774 consensus_queue.cc:260] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } }
I20251028 21:12:45.452524 22774 raft_consensus.cc:3060] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 2 FOLLOWER]: Advancing to term 3
I20251028 21:12:45.572360 22727 raft_consensus.cc:1275] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Refusing update from remote peer db3e1355336c49e0a1de88a006287cd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251028 21:12:45.572613 22774 consensus_queue.cc:1048] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.573726 22564 raft_consensus.cc:1275] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Refusing update from remote peer db3e1355336c49e0a1de88a006287cd9: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251028 21:12:45.573932 22788 consensus_queue.cc:1048] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.574007 22788 consensus_queue.cc:1243] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Peer f4bf70d569404139a73af4bc15f4cac8 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251028 21:12:45.574271 22564 pending_rounds.cc:85] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Aborting all ops after (but not including) 0
I20251028 21:12:45.574317 22564 pending_rounds.cc:107] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251028 21:12:45.574348 22564 raft_consensus.cc:2889] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251028 21:12:45.647151 22564 raft_consensus.cc:1275] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Refusing update from remote peer 0a66afdb6b2e4b0c873b732c6bb6c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251028 21:12:45.647392 22792 consensus_queue.cc:1048] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.648665 22645 raft_consensus.cc:1275] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Refusing update from remote peer 0a66afdb6b2e4b0c873b732c6bb6c3a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251028 21:12:45.648888 22779 consensus_queue.cc:1048] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:45.648976 22779 consensus_queue.cc:1243] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Peer db3e1355336c49e0a1de88a006287cd9 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251028 21:12:45.649149 22645 pending_rounds.cc:85] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Aborting all ops after (but not including) 0
I20251028 21:12:45.649181 22645 pending_rounds.cc:107] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251028 21:12:45.649205 22645 raft_consensus.cc:2889] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251028 21:12:46.219211 22727 consensus_queue.cc:237] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.219735 22564 raft_consensus.cc:1275] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Refusing update from remote peer 0a66afdb6b2e4b0c873b732c6bb6c3a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251028 21:12:46.219861 22645 raft_consensus.cc:1275] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Refusing update from remote peer 0a66afdb6b2e4b0c873b732c6bb6c3a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251028 21:12:46.220016 22782 consensus_queue.cc:1048] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:46.220136 22779 consensus_queue.cc:1048] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:46.220726 22779 raft_consensus.cc:2955] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.220816 22645 raft_consensus.cc:2955] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.220873 22564 raft_consensus.cc:2955] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.221802 22095 catalog_manager.cc:5649] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 reported cstate change: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New cstate: current_term: 3 leader_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:46.221807 22727 consensus_queue.cc:237] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } attrs { replace: true } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.222468 22645 raft_consensus.cc:1275] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Refusing update from remote peer 0a66afdb6b2e4b0c873b732c6bb6c3a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251028 21:12:46.222678 22564 raft_consensus.cc:1275] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Refusing update from remote peer 0a66afdb6b2e4b0c873b732c6bb6c3a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251028 21:12:46.222657 22782 consensus_queue.cc:1048] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:46.222837 22782 consensus_queue.cc:1048] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251028 21:12:46.223078 22689 consensus_peers.cc:597] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 -> Peer 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4:35151): Couldn't send request to peer 55b99ae813b64732ba32947d7d4ee9d9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d46a3a73f9184bd29dc76964c72570c4. This is attempt 1: this message will repeat every 5th retry.
W20251028 21:12:46.223138 22689 consensus_peers.cc:597] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 -> Peer 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4:35151): Couldn't send request to peer 55b99ae813b64732ba32947d7d4ee9d9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e18d869e511a421d9955701ea3f4d964. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:46.223487 22782 raft_consensus.cc:2955] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } attrs { replace: true } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.223636 22564 raft_consensus.cc:2955] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } attrs { replace: true } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.224422 22094 catalog_manager.cc:5649] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 reported cstate change: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New cstate: current_term: 3 leader_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } attrs { replace: true } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.224525 22644 raft_consensus.cc:2955] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } attrs { replace: true } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.224606 22645 consensus_queue.cc:237] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.224977 22564 raft_consensus.cc:1275] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Refusing update from remote peer db3e1355336c49e0a1de88a006287cd9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251028 21:12:46.225008 22727 raft_consensus.cc:1275] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Refusing update from remote peer db3e1355336c49e0a1de88a006287cd9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251028 21:12:46.225157 22774 consensus_queue.cc:1048] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:46.225232 22765 consensus_queue.cc:1048] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:46.225718 22774 raft_consensus.cc:2955] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.225836 22564 raft_consensus.cc:2955] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.226282 22727 raft_consensus.cc:2955] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
I20251028 21:12:46.227368 22095 catalog_manager.cc:5649] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 reported cstate change: config changed from index -1 to 3, NON_VOTER 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4) added. New cstate: current_term: 3 leader_uuid: "db3e1355336c49e0a1de88a006287cd9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } } }
W20251028 21:12:46.228322 22608 consensus_peers.cc:597] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 -> Peer 55b99ae813b64732ba32947d7d4ee9d9 (127.15.205.4:35151): Couldn't send request to peer 55b99ae813b64732ba32947d7d4ee9d9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3761d146ab3243c99e447333f8cd496b. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:46.623070 22805 ts_tablet_manager.cc:933] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: Initiating tablet copy from peer 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:46.623255 22805 tablet_copy_client.cc:323] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:36905
I20251028 21:12:46.626374 22737 tablet_copy_service.cc:140] P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Received BeginTabletCopySession request for tablet e18d869e511a421d9955701ea3f4d964 from peer 55b99ae813b64732ba32947d7d4ee9d9 ({username='slave'} at 127.0.0.1:57538)
I20251028 21:12:46.626428 22737 tablet_copy_service.cc:161] P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Beginning new tablet copy session on tablet e18d869e511a421d9955701ea3f4d964 from peer 55b99ae813b64732ba32947d7d4ee9d9 at {username='slave'} at 127.0.0.1:57538: session id = 55b99ae813b64732ba32947d7d4ee9d9-e18d869e511a421d9955701ea3f4d964
I20251028 21:12:46.626890 22737 tablet_copy_source_session.cc:215] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:46.627197 22805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e18d869e511a421d9955701ea3f4d964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:46.628607 22805 tablet_copy_client.cc:806] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:46.628794 22805 tablet_copy_client.cc:670] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:46.629319 22805 tablet_copy_client.cc:538] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:46.630003 22805 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap starting.
I20251028 21:12:46.631369 22805 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:46.631543 22805 tablet_bootstrap.cc:492] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap complete.
I20251028 21:12:46.631628 22805 ts_tablet_manager.cc:1403] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20251028 21:12:46.631769 22805 raft_consensus.cc:359] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } attrs { replace: true } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.631845 22805 raft_consensus.cc:740] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 55b99ae813b64732ba32947d7d4ee9d9, State: Initialized, Role: LEARNER
I20251028 21:12:46.631883 22805 consensus_queue.cc:260] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } attrs { replace: true } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.632036 22805 ts_tablet_manager.cc:1434] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:46.632256 22737 tablet_copy_service.cc:342] P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Request end of tablet copy session 55b99ae813b64732ba32947d7d4ee9d9-e18d869e511a421d9955701ea3f4d964 received from {username='slave'} at 127.0.0.1:57538
I20251028 21:12:46.632304 22737 tablet_copy_service.cc:434] P 0a66afdb6b2e4b0c873b732c6bb6c3a2: ending tablet copy session 55b99ae813b64732ba32947d7d4ee9d9-e18d869e511a421d9955701ea3f4d964 on tablet e18d869e511a421d9955701ea3f4d964 with peer 55b99ae813b64732ba32947d7d4ee9d9
I20251028 21:12:46.783099 22805 ts_tablet_manager.cc:933] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: Initiating tablet copy from peer db3e1355336c49e0a1de88a006287cd9 (127.15.205.2:44171)
I20251028 21:12:46.783349 22805 tablet_copy_client.cc:323] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:44171
I20251028 21:12:46.786520 22656 tablet_copy_service.cc:140] P db3e1355336c49e0a1de88a006287cd9: Received BeginTabletCopySession request for tablet 3761d146ab3243c99e447333f8cd496b from peer 55b99ae813b64732ba32947d7d4ee9d9 ({username='slave'} at 127.0.0.1:40386)
I20251028 21:12:46.786582 22656 tablet_copy_service.cc:161] P db3e1355336c49e0a1de88a006287cd9: Beginning new tablet copy session on tablet 3761d146ab3243c99e447333f8cd496b from peer 55b99ae813b64732ba32947d7d4ee9d9 at {username='slave'} at 127.0.0.1:40386: session id = 55b99ae813b64732ba32947d7d4ee9d9-3761d146ab3243c99e447333f8cd496b
I20251028 21:12:46.787112 22656 tablet_copy_source_session.cc:215] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:46.787421 22805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3761d146ab3243c99e447333f8cd496b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:46.788679 22805 tablet_copy_client.cc:806] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:46.788800 22805 tablet_copy_client.cc:670] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:46.789207 22805 tablet_copy_client.cc:538] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:46.789847 22805 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap starting.
I20251028 21:12:46.791231 22805 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:46.791425 22805 tablet_bootstrap.cc:492] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap complete.
I20251028 21:12:46.791519 22805 ts_tablet_manager.cc:1403] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20251028 21:12:46.791656 22805 raft_consensus.cc:359] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.791716 22805 raft_consensus.cc:740] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 55b99ae813b64732ba32947d7d4ee9d9, State: Initialized, Role: LEARNER
I20251028 21:12:46.791769 22805 consensus_queue.cc:260] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } } peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.792003 22805 ts_tablet_manager.cc:1434] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:46.792164 22656 tablet_copy_service.cc:342] P db3e1355336c49e0a1de88a006287cd9: Request end of tablet copy session 55b99ae813b64732ba32947d7d4ee9d9-3761d146ab3243c99e447333f8cd496b received from {username='slave'} at 127.0.0.1:40386
I20251028 21:12:46.792224 22656 tablet_copy_service.cc:434] P db3e1355336c49e0a1de88a006287cd9: ending tablet copy session 55b99ae813b64732ba32947d7d4ee9d9-3761d146ab3243c99e447333f8cd496b on tablet 3761d146ab3243c99e447333f8cd496b with peer 55b99ae813b64732ba32947d7d4ee9d9
I20251028 21:12:46.816438 22805 ts_tablet_manager.cc:933] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: Initiating tablet copy from peer 0a66afdb6b2e4b0c873b732c6bb6c3a2 (127.15.205.3:36905)
I20251028 21:12:46.816568 22805 tablet_copy_client.cc:323] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:36905
I20251028 21:12:46.816727 22737 tablet_copy_service.cc:140] P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Received BeginTabletCopySession request for tablet d46a3a73f9184bd29dc76964c72570c4 from peer 55b99ae813b64732ba32947d7d4ee9d9 ({username='slave'} at 127.0.0.1:57538)
I20251028 21:12:46.816782 22737 tablet_copy_service.cc:161] P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Beginning new tablet copy session on tablet d46a3a73f9184bd29dc76964c72570c4 from peer 55b99ae813b64732ba32947d7d4ee9d9 at {username='slave'} at 127.0.0.1:57538: session id = 55b99ae813b64732ba32947d7d4ee9d9-d46a3a73f9184bd29dc76964c72570c4
I20251028 21:12:46.817279 22737 tablet_copy_source_session.cc:215] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:46.817531 22805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46a3a73f9184bd29dc76964c72570c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:46.818841 22805 tablet_copy_client.cc:806] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:46.818986 22805 tablet_copy_client.cc:670] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:46.819423 22805 tablet_copy_client.cc:538] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:46.820025 22805 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap starting.
I20251028 21:12:46.821345 22805 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:46.821475 22805 tablet_bootstrap.cc:492] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: Bootstrap complete.
I20251028 21:12:46.821537 22805 ts_tablet_manager.cc:1403] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 21:12:46.821664 22805 raft_consensus.cc:359] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.821735 22805 raft_consensus.cc:740] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 55b99ae813b64732ba32947d7d4ee9d9, State: Initialized, Role: LEARNER
I20251028 21:12:46.821768 22805 consensus_queue.cc:260] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "db3e1355336c49e0a1de88a006287cd9" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44171 } } peers { permanent_uuid: "0a66afdb6b2e4b0c873b732c6bb6c3a2" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 36905 } } peers { permanent_uuid: "f4bf70d569404139a73af4bc15f4cac8" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41039 } attrs { replace: true } } peers { permanent_uuid: "55b99ae813b64732ba32947d7d4ee9d9" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 35151 } attrs { promote: true } }
I20251028 21:12:46.821862 22805 ts_tablet_manager.cc:1434] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:46.822173 22737 tablet_copy_service.cc:342] P 0a66afdb6b2e4b0c873b732c6bb6c3a2: Request end of tablet copy session 55b99ae813b64732ba32947d7d4ee9d9-d46a3a73f9184bd29dc76964c72570c4 received from {username='slave'} at 127.0.0.1:57538
I20251028 21:12:46.822228 22737 tablet_copy_service.cc:434] P 0a66afdb6b2e4b0c873b732c6bb6c3a2: ending tablet copy session 55b99ae813b64732ba32947d7d4ee9d9-d46a3a73f9184bd29dc76964c72570c4 on tablet d46a3a73f9184bd29dc76964c72570c4 with peer 55b99ae813b64732ba32947d7d4ee9d9
I20251028 21:12:47.126142 22469 raft_consensus.cc:1217] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20251028 21:12:47.199296 22469 raft_consensus.cc:1217] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20251028 21:12:47.389071 22469 raft_consensus.cc:1217] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
I20251028 21:12:47.426926 16180 tablet_server.cc:178] TabletServer@127.15.205.1:41039 shutting down...
W20251028 21:12:47.429322 22605 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
W20251028 21:12:47.429620 22605 consensus_peers.cc:597] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 -> Peer f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039): Couldn't send request to peer f4bf70d569404139a73af4bc15f4cac8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:47.429632 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:47.429738 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.429798 16180 raft_consensus.cc:2243] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:47.429847 16180 raft_consensus.cc:2272] T d46a3a73f9184bd29dc76964c72570c4 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.430017 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.430068 16180 raft_consensus.cc:2243] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:47.430128 16180 raft_consensus.cc:2272] T e18d869e511a421d9955701ea3f4d964 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.430263 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.430312 16180 raft_consensus.cc:2243] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:47.430347 16180 raft_consensus.cc:2272] T 3761d146ab3243c99e447333f8cd496b P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.430460 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.430501 16180 raft_consensus.cc:2243] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:47.430539 16180 raft_consensus.cc:2272] T de68b0b504164887810816a7a59a1ba0 P f4bf70d569404139a73af4bc15f4cac8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.442308 16180 tablet_server.cc:195] TabletServer@127.15.205.1:41039 shutdown complete.
I20251028 21:12:47.443475 16180 tablet_server.cc:178] TabletServer@127.15.205.2:44171 shutting down...
W20251028 21:12:47.444321 22144 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:12:47.444399 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:12:47.446640 22605 consensus_peers.cc:597] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 -> Peer f4bf70d569404139a73af4bc15f4cac8 (127.15.205.1:41039): Couldn't send request to peer f4bf70d569404139a73af4bc15f4cac8. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:41039: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:47.446790 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:47.446909 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.446969 16180 raft_consensus.cc:2243] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:47.447022 16180 raft_consensus.cc:2272] T d46a3a73f9184bd29dc76964c72570c4 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.447182 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.447223 16180 raft_consensus.cc:2243] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:47.447319 16180 raft_consensus.cc:2272] T e18d869e511a421d9955701ea3f4d964 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.447542 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.447592 16180 raft_consensus.cc:2243] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 3 LEADER]: Raft consensus shutting down.
I20251028 21:12:47.447707 16180 raft_consensus.cc:2272] T 3761d146ab3243c99e447333f8cd496b P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.447834 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.447875 16180 raft_consensus.cc:2243] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 3 LEADER]: Raft consensus shutting down.
I20251028 21:12:47.447949 16180 raft_consensus.cc:2272] T de68b0b504164887810816a7a59a1ba0 P db3e1355336c49e0a1de88a006287cd9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.459816 16180 tablet_server.cc:195] TabletServer@127.15.205.2:44171 shutdown complete.
I20251028 21:12:47.460826 16180 tablet_server.cc:178] TabletServer@127.15.205.3:36905 shutting down...
W20251028 21:12:47.461237 22144 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:12:47.461381 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:12:47.461716 22144 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251028 21:12:47.461761 22144 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251028 21:12:47.463774 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:47.463862 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.463924 16180 raft_consensus.cc:2243] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 LEADER]: Raft consensus shutting down.
I20251028 21:12:47.463985 16180 raft_consensus.cc:2272] T d46a3a73f9184bd29dc76964c72570c4 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.464097 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.464125 16180 raft_consensus.cc:2243] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 LEADER]: Raft consensus shutting down.
I20251028 21:12:47.464170 16180 raft_consensus.cc:2272] T e18d869e511a421d9955701ea3f4d964 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.464244 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.464272 16180 raft_consensus.cc:2243] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:47.464298 16180 raft_consensus.cc:2272] T 3761d146ab3243c99e447333f8cd496b P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.464363 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:47.464386 16180 raft_consensus.cc:2243] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:47.464407 16180 raft_consensus.cc:2272] T de68b0b504164887810816a7a59a1ba0 P 0a66afdb6b2e4b0c873b732c6bb6c3a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.475889 16180 tablet_server.cc:195] TabletServer@127.15.205.3:36905 shutdown complete.
I20251028 21:12:47.476802 16180 tablet_server.cc:178] TabletServer@127.15.205.4:0 shutting down...
I20251028 21:12:47.479538 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:47.479651 16180 tablet_replica.cc:333] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9: stopping tablet replica
I20251028 21:12:47.479702 16180 raft_consensus.cc:2243] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Raft consensus shutting down.
I20251028 21:12:47.479744 16180 raft_consensus.cc:2272] T d46a3a73f9184bd29dc76964c72570c4 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Raft consensus is shut down!
I20251028 21:12:47.479864 16180 tablet_replica.cc:333] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9: stopping tablet replica
I20251028 21:12:47.479902 16180 raft_consensus.cc:2243] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Raft consensus shutting down.
I20251028 21:12:47.479933 16180 raft_consensus.cc:2272] T e18d869e511a421d9955701ea3f4d964 P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Raft consensus is shut down!
I20251028 21:12:47.480036 16180 tablet_replica.cc:333] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9: stopping tablet replica
I20251028 21:12:47.480060 16180 raft_consensus.cc:2243] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Raft consensus shutting down.
I20251028 21:12:47.480090 16180 raft_consensus.cc:2272] T 3761d146ab3243c99e447333f8cd496b P 55b99ae813b64732ba32947d7d4ee9d9 [term 3 LEARNER]: Raft consensus is shut down!
I20251028 21:12:47.491375 16180 tablet_server.cc:195] TabletServer@127.15.205.4:0 shutdown complete.
I20251028 21:12:47.492110 16180 master.cc:561] Master@127.15.205.62:43919 shutting down...
I20251028 21:12:47.494331 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:47.494385 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:47.494416 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 786a596dedd54a0aad3dd82e7ffe9811: stopping tablet replica
I20251028 21:12:47.502557 16180 master.cc:583] Master@127.15.205.62:43919 shutdown complete.
[ OK ] AutoRebalancerTest.TestHandlingFailedTservers (11331 ms)
[ RUN ] AutoRebalancerTest.TestDeletedTables
I20251028 21:12:47.509959 16180 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.205.62:33985
I20251028 21:12:47.510083 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:47.511237 22822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:47.511334 22825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:47.511389 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:47.511350 22823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:47.511629 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:47.511670 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:47.511688 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685967511689 us; error 0 us; skew 500 ppm
I20251028 21:12:47.512183 16180 webserver.cc:492] Webserver started at http://127.15.205.62:46413/ using document root <none> and password file <none>
I20251028 21:12:47.512259 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:47.512302 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:47.512336 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:47.512538 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/master-0-root/instance:
uuid: "6b8026167662446e846ba07fa0042298"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:47.513253 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:47.513631 22830 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.513743 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.513783 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/master-0-root
uuid: "6b8026167662446e846ba07fa0042298"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:47.513828 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:47.532963 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:47.533115 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:47.535199 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.62:33985
I20251028 21:12:47.538046 22882 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.62:33985 every 8 connection(s)
I20251028 21:12:47.538121 22883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:47.539086 22883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: Bootstrap starting.
I20251028 21:12:47.539342 22883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:47.539816 22883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: No bootstrap required, opened a new log
I20251028 21:12:47.539929 22883 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8026167662446e846ba07fa0042298" member_type: VOTER }
I20251028 21:12:47.539969 22883 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:47.539994 22883 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b8026167662446e846ba07fa0042298, State: Initialized, Role: FOLLOWER
I20251028 21:12:47.540035 22883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8026167662446e846ba07fa0042298" member_type: VOTER }
I20251028 21:12:47.540067 22883 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251028 21:12:47.540091 22883 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251028 21:12:47.540114 22883 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:47.540549 22883 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8026167662446e846ba07fa0042298" member_type: VOTER }
I20251028 21:12:47.540611 22883 leader_election.cc:304] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b8026167662446e846ba07fa0042298; no voters:
I20251028 21:12:47.540711 22883 leader_election.cc:290] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251028 21:12:47.540761 22886 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:47.540845 22883 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 21:12:47.540853 22886 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 1 LEADER]: Becoming Leader. State: Replica: 6b8026167662446e846ba07fa0042298, State: Running, Role: LEADER
I20251028 21:12:47.541006 22886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8026167662446e846ba07fa0042298" member_type: VOTER }
I20251028 21:12:47.541206 22887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b8026167662446e846ba07fa0042298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8026167662446e846ba07fa0042298" member_type: VOTER } }
I20251028 21:12:47.541208 22888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b8026167662446e846ba07fa0042298. Latest consensus state: current_term: 1 leader_uuid: "6b8026167662446e846ba07fa0042298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8026167662446e846ba07fa0042298" member_type: VOTER } }
I20251028 21:12:47.541360 22888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:47.541517 22894 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 21:12:47.541517 22887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [sys.catalog]: This master's current role is: LEADER
I20251028 21:12:47.541702 22894 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 21:12:47.542285 16180 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251028 21:12:47.542335 22894 catalog_manager.cc:1357] Generated new cluster ID: 6bd58fc008c74734bafb4a670774f01c
I20251028 21:12:47.542397 22894 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 21:12:47.551031 22894 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 21:12:47.551451 22894 catalog_manager.cc:1514] Loading token signing keys...
I20251028 21:12:47.559335 22894 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: Generated new TSK 0
I20251028 21:12:47.559397 22894 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 21:12:47.574064 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:12:47.575307 16180 server_base.cc:1047] running on GCE node
W20251028 21:12:47.575357 22907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:47.575358 22905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:47.575412 22904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:47.575551 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:47.575589 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:47.575603 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685967575602 us; error 0 us; skew 500 ppm
I20251028 21:12:47.576059 16180 webserver.cc:492] Webserver started at http://127.15.205.1:38883/ using document root <none> and password file <none>
I20251028 21:12:47.576120 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:47.576157 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:47.576188 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:47.576398 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root/instance:
uuid: "3a25731407814b3bb1a4125abce11bbd"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:47.577095 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:47.577455 22912 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.577564 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.577607 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "3a25731407814b3bb1a4125abce11bbd"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:47.577643 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:47.588079 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:47.588224 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:47.588320 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:47.588476 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:47.588501 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.588533 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:47.588557 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.591045 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:41405
I20251028 21:12:47.591073 22975 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:41405 every 8 connection(s)
I20251028 21:12:47.591372 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:47.592414 22980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:47.592562 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:47.592447 22981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:47.592684 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:47.592794 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:47.592821 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:47.592837 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685967592837 us; error 0 us; skew 500 ppm
I20251028 21:12:47.593338 16180 webserver.cc:492] Webserver started at http://127.15.205.2:40153/ using document root <none> and password file <none>
I20251028 21:12:47.593427 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:47.593473 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:47.593518 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:47.593731 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root/instance:
uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:47.594704 22976 heartbeater.cc:344] Connected to a master server at 127.15.205.62:33985
I20251028 21:12:47.594761 22976 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:47.594831 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:47.594877 22976 heartbeater.cc:507] Master 127.15.205.62:33985 requested a full tablet report, sending...
I20251028 21:12:47.595109 22847 ts_manager.cc:194] Registered new tserver with Master: 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405)
I20251028 21:12:47.595335 22988 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.595465 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:47.595511 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root
uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:47.595551 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:47.595762 22847 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50518
I20251028 21:12:47.608742 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:47.608924 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:47.609040 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:47.609201 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:47.609228 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.609263 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:47.609288 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.611781 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.2:44217
I20251028 21:12:47.611847 23051 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.2:44217 every 8 connection(s)
I20251028 21:12:47.612123 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:47.613113 23056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:47.613179 23058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:47.613123 23055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:47.613613 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:47.613709 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:47.613736 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:47.613754 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685967613754 us; error 0 us; skew 500 ppm
I20251028 21:12:47.614297 16180 webserver.cc:492] Webserver started at http://127.15.205.3:37735/ using document root <none> and password file <none>
I20251028 21:12:47.614378 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:47.614420 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:47.614465 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:47.614778 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root/instance:
uuid: "bb4ab48d916944119f57263513c12fb8"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:47.614971 23052 heartbeater.cc:344] Connected to a master server at 127.15.205.62:33985
I20251028 21:12:47.615028 23052 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:47.615108 23052 heartbeater.cc:507] Master 127.15.205.62:33985 requested a full tablet report, sending...
I20251028 21:12:47.615321 22847 ts_manager.cc:194] Registered new tserver with Master: 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217)
I20251028 21:12:47.615585 22847 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50534
I20251028 21:12:47.615762 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:47.616181 23063 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.616287 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.616326 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root
uuid: "bb4ab48d916944119f57263513c12fb8"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:47.616367 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:47.621819 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:47.621978 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:47.622113 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:47.622252 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:47.622277 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.622300 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:47.622325 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:47.624846 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.3:45395
I20251028 21:12:47.624873 23126 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.3:45395 every 8 connection(s)
I20251028 21:12:47.627336 23127 heartbeater.cc:344] Connected to a master server at 127.15.205.62:33985
I20251028 21:12:47.627385 23127 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:47.627445 23127 heartbeater.cc:507] Master 127.15.205.62:33985 requested a full tablet report, sending...
I20251028 21:12:47.627615 22847 ts_manager.cc:194] Registered new tserver with Master: bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:47.627924 22847 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50544
I20251028 21:12:47.628298 16180 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003288771s
I20251028 21:12:48.596342 22976 heartbeater.cc:499] Master 127.15.205.62:33985 was elected leader, sending a full tablet report...
I20251028 21:12:48.616171 23052 heartbeater.cc:499] Master 127.15.205.62:33985 was elected leader, sending a full tablet report...
I20251028 21:12:48.628357 23127 heartbeater.cc:499] Master 127.15.205.62:33985 was elected leader, sending a full tablet report...
I20251028 21:12:48.653853 16180 test_util.cc:276] Using random seed: 2052984347
I20251028 21:12:48.664866 22847 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:12:48.665012 22847 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:12:48.669785 23015 tablet_service.cc:1505] Processing CreateTablet for tablet e6d1954a45164b32b866a241b9487e59 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:48.669785 23016 tablet_service.cc:1505] Processing CreateTablet for tablet 9f9a28ac41a04f3990922179532c2ca9 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:48.669958 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6d1954a45164b32b866a241b9487e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.670061 22940 tablet_service.cc:1505] Processing CreateTablet for tablet 9f9a28ac41a04f3990922179532c2ca9 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:48.670060 23016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9a28ac41a04f3990922179532c2ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.670217 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9a28ac41a04f3990922179532c2ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.671350 23147 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:48.671465 23014 tablet_service.cc:1505] Processing CreateTablet for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:48.671491 22937 tablet_service.cc:1505] Processing CreateTablet for tablet 9b3c763a0a8d441284b698fe0949090d (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:48.671556 23014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1aa778fb70c461fa81bcc7cd2dc3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.671576 22937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3c763a0a8d441284b698fe0949090d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.672372 23147 tablet_bootstrap.cc:654] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.672683 23091 tablet_service.cc:1505] Processing CreateTablet for tablet 9f9a28ac41a04f3990922179532c2ca9 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:48.672715 23090 tablet_service.cc:1505] Processing CreateTablet for tablet e6d1954a45164b32b866a241b9487e59 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:48.672777 23091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f9a28ac41a04f3990922179532c2ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.671394 22938 tablet_service.cc:1505] Processing CreateTablet for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:48.672838 23089 tablet_service.cc:1505] Processing CreateTablet for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:48.672904 22938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1aa778fb70c461fa81bcc7cd2dc3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.671362 22939 tablet_service.cc:1505] Processing CreateTablet for tablet e6d1954a45164b32b866a241b9487e59 (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:48.672920 23089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1aa778fb70c461fa81bcc7cd2dc3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.672972 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6d1954a45164b32b866a241b9487e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.673414 23148 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e: Bootstrap starting.
I20251028 21:12:48.673804 23148 tablet_bootstrap.cc:654] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.671376 23013 tablet_service.cc:1505] Processing CreateTablet for tablet 9b3c763a0a8d441284b698fe0949090d (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:48.674132 23088 tablet_service.cc:1505] Processing CreateTablet for tablet 9b3c763a0a8d441284b698fe0949090d (DEFAULT_TABLE table=test-workload [id=cd63739fa16141fb9abb319114098251]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:48.674162 23013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3c763a0a8d441284b698fe0949090d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.674206 23088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3c763a0a8d441284b698fe0949090d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.674217 23147 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: No bootstrap required, opened a new log
I20251028 21:12:48.674285 23147 ts_tablet_manager.cc:1403] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:12:48.674501 23147 raft_consensus.cc:359] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.674569 23147 raft_consensus.cc:385] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.674595 23147 raft_consensus.cc:740] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.674649 23147 consensus_queue.cc:260] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.674791 23147 ts_tablet_manager.cc:1434] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.674844 23147 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:48.675242 23090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6d1954a45164b32b866a241b9487e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:48.675320 23147 tablet_bootstrap.cc:654] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.675820 23148 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e: No bootstrap required, opened a new log
I20251028 21:12:48.675891 23148 ts_tablet_manager.cc:1403] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:48.675902 23147 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: No bootstrap required, opened a new log
I20251028 21:12:48.675976 23147 ts_tablet_manager.cc:1403] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:48.676067 23148 raft_consensus.cc:359] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.676088 23147 raft_consensus.cc:359] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.676116 23148 raft_consensus.cc:385] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.676126 23147 raft_consensus.cc:385] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.676141 23148 raft_consensus.cc:740] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.676151 23147 raft_consensus.cc:740] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.676193 23147 consensus_queue.cc:260] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.676183 23148 consensus_queue.cc:260] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.676275 23147 ts_tablet_manager.cc:1434] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.676332 23147 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:48.676333 23148 ts_tablet_manager.cc:1434] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.676482 23148 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: Bootstrap starting.
I20251028 21:12:48.676656 23150 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8: Bootstrap starting.
I20251028 21:12:48.676713 23147 tablet_bootstrap.cc:654] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.676827 23148 tablet_bootstrap.cc:654] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.677030 23150 tablet_bootstrap.cc:654] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.677253 23147 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: No bootstrap required, opened a new log
I20251028 21:12:48.677299 23147 ts_tablet_manager.cc:1403] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:48.677430 23147 raft_consensus.cc:359] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.677484 23147 raft_consensus.cc:385] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.677507 23147 raft_consensus.cc:740] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.677505 23150 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8: No bootstrap required, opened a new log
I20251028 21:12:48.677575 23150 ts_tablet_manager.cc:1403] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:48.677560 23147 consensus_queue.cc:260] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.677642 23147 ts_tablet_manager.cc:1434] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.677690 23147 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:48.677708 23150 raft_consensus.cc:359] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.677752 23150 raft_consensus.cc:385] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.677764 23150 raft_consensus.cc:740] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.677832 23150 consensus_queue.cc:260] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.677991 23150 ts_tablet_manager.cc:1434] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.678071 23150 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8: Bootstrap starting.
I20251028 21:12:48.678099 23147 tablet_bootstrap.cc:654] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.678606 23150 tablet_bootstrap.cc:654] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.678699 23147 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: No bootstrap required, opened a new log
I20251028 21:12:48.678747 23147 ts_tablet_manager.cc:1403] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:48.678876 23147 raft_consensus.cc:359] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.678936 23147 raft_consensus.cc:385] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.678961 23147 raft_consensus.cc:740] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.679006 23147 consensus_queue.cc:260] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.679090 23147 ts_tablet_manager.cc:1434] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.679148 23148 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: No bootstrap required, opened a new log
I20251028 21:12:48.679152 23150 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8: No bootstrap required, opened a new log
I20251028 21:12:48.679201 23148 ts_tablet_manager.cc:1403] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:12:48.679208 23150 ts_tablet_manager.cc:1403] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:48.679409 23148 raft_consensus.cc:359] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.679409 23150 raft_consensus.cc:359] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.679492 23148 raft_consensus.cc:385] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.679500 23150 raft_consensus.cc:385] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.679525 23148 raft_consensus.cc:740] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.679538 23150 raft_consensus.cc:740] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.679566 23148 consensus_queue.cc:260] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.679581 23150 consensus_queue.cc:260] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.679644 23150 ts_tablet_manager.cc:1434] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.679713 23150 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: Bootstrap starting.
I20251028 21:12:48.680063 23150 tablet_bootstrap.cc:654] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.679644 23148 ts_tablet_manager.cc:1434] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.680426 23148 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e: Bootstrap starting.
I20251028 21:12:48.680502 23150 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: No bootstrap required, opened a new log
I20251028 21:12:48.680536 23150 ts_tablet_manager.cc:1403] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:48.680660 23150 raft_consensus.cc:359] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.680707 23150 raft_consensus.cc:385] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.680719 23150 raft_consensus.cc:740] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.680755 23150 consensus_queue.cc:260] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.680905 23150 ts_tablet_manager.cc:1434] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.680977 23150 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8: Bootstrap starting.
I20251028 21:12:48.681340 23150 tablet_bootstrap.cc:654] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.680759 23148 tablet_bootstrap.cc:654] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.681883 23150 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8: No bootstrap required, opened a new log
I20251028 21:12:48.681937 23150 ts_tablet_manager.cc:1403] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:48.682073 23150 raft_consensus.cc:359] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.682139 23150 raft_consensus.cc:385] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.682188 23150 raft_consensus.cc:740] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.682250 23150 consensus_queue.cc:260] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.682341 23150 ts_tablet_manager.cc:1434] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.682143 23148 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e: No bootstrap required, opened a new log
I20251028 21:12:48.682430 23148 ts_tablet_manager.cc:1403] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:48.682564 23148 raft_consensus.cc:359] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.682617 23148 raft_consensus.cc:385] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.682637 23148 raft_consensus.cc:740] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.682682 23148 consensus_queue.cc:260] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.682760 23148 ts_tablet_manager.cc:1434] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.682819 23148 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e: Bootstrap starting.
I20251028 21:12:48.683187 23148 tablet_bootstrap.cc:654] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:48.683678 23148 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e: No bootstrap required, opened a new log
I20251028 21:12:48.683719 23148 ts_tablet_manager.cc:1403] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:48.683827 23148 raft_consensus.cc:359] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.683871 23148 raft_consensus.cc:385] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:48.683893 23148 raft_consensus.cc:740] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Initialized, Role: FOLLOWER
I20251028 21:12:48.683940 23148 consensus_queue.cc:260] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.684046 23148 ts_tablet_manager.cc:1434] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:48.703290 23152 raft_consensus.cc:493] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:48.703342 23152 raft_consensus.cc:515] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.703487 23152 leader_election.cc:290] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:48.705602 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" is_pre_election: true
I20251028 21:12:48.705669 23026 raft_consensus.cc:2468] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:48.705653 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8" is_pre_election: true
I20251028 21:12:48.705818 23101 raft_consensus.cc:2468] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:48.705942 22915 leader_election.cc:304] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, 7b24afaeee5d4b2388b84d94ea28a92e; no voters:
I20251028 21:12:48.706043 23152 raft_consensus.cc:2804] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:48.706089 23152 raft_consensus.cc:493] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:48.706105 23152 raft_consensus.cc:3060] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.706657 23152 raft_consensus.cc:515] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.706769 23152 leader_election.cc:290] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Requested vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:48.706897 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
I20251028 21:12:48.706938 23026 raft_consensus.cc:3060] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.707463 23026 raft_consensus.cc:2468] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:48.707547 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8"
I20251028 21:12:48.707597 22915 leader_election.cc:304] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, 7b24afaeee5d4b2388b84d94ea28a92e; no voters:
I20251028 21:12:48.707603 23101 raft_consensus.cc:3060] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.708089 23101 raft_consensus.cc:2468] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:48.708184 23152 raft_consensus.cc:2804] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:48.708312 23152 raft_consensus.cc:697] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Becoming Leader. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Running, Role: LEADER
I20251028 21:12:48.708345 23152 consensus_queue.cc:237] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:48.708766 22847 catalog_manager.cc:5649] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a25731407814b3bb1a4125abce11bbd (127.15.205.1). New cstate: current_term: 1 leader_uuid: "3a25731407814b3bb1a4125abce11bbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:48.758589 23162 raft_consensus.cc:493] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:48.758652 23162 raft_consensus.cc:515] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.758797 23162 leader_election.cc:290] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405), 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217)
I20251028 21:12:48.761106 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a1aa778fb70c461fa81bcc7cd2dc3db1" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" is_pre_election: true
I20251028 21:12:48.761199 23026 raft_consensus.cc:2468] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 0.
I20251028 21:12:48.761173 22950 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a1aa778fb70c461fa81bcc7cd2dc3db1" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd" is_pre_election: true
I20251028 21:12:48.761274 22950 raft_consensus.cc:2468] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 0.
I20251028 21:12:48.761344 23066 leader_election.cc:304] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b24afaeee5d4b2388b84d94ea28a92e, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:48.761423 23162 raft_consensus.cc:2804] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:48.761464 23162 raft_consensus.cc:493] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:48.761478 23162 raft_consensus.cc:3060] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.761961 23162 raft_consensus.cc:515] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.762054 23162 leader_election.cc:290] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405), 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217)
I20251028 21:12:48.762185 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a1aa778fb70c461fa81bcc7cd2dc3db1" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
I20251028 21:12:48.762183 22950 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a1aa778fb70c461fa81bcc7cd2dc3db1" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd"
I20251028 21:12:48.762253 23026 raft_consensus.cc:3060] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.762259 22950 raft_consensus.cc:3060] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.762760 23026 raft_consensus.cc:2468] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 1.
I20251028 21:12:48.762817 22950 raft_consensus.cc:2468] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 1.
I20251028 21:12:48.762933 23066 leader_election.cc:304] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b24afaeee5d4b2388b84d94ea28a92e, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:48.763015 23162 raft_consensus.cc:2804] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:48.763062 23162 raft_consensus.cc:697] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Becoming Leader. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Running, Role: LEADER
I20251028 21:12:48.763094 23162 consensus_queue.cc:237] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.763541 22847 catalog_manager.cc:5649] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb4ab48d916944119f57263513c12fb8 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:48.771217 23154 raft_consensus.cc:493] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:48.771309 23154 raft_consensus.cc:515] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.771467 23154 leader_election.cc:290] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:48.773653 22950 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b3c763a0a8d441284b698fe0949090d" candidate_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd" is_pre_election: true
I20251028 21:12:48.773722 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b3c763a0a8d441284b698fe0949090d" candidate_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8" is_pre_election: true
I20251028 21:12:48.773726 22950 raft_consensus.cc:2468] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b24afaeee5d4b2388b84d94ea28a92e in term 0.
I20251028 21:12:48.773804 23101 raft_consensus.cc:2468] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b24afaeee5d4b2388b84d94ea28a92e in term 0.
I20251028 21:12:48.773910 22992 leader_election.cc:304] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, 7b24afaeee5d4b2388b84d94ea28a92e; no voters:
I20251028 21:12:48.774003 23154 raft_consensus.cc:2804] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:48.774045 23154 raft_consensus.cc:493] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:48.774070 23154 raft_consensus.cc:3060] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.774511 23154 raft_consensus.cc:515] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.774593 23154 leader_election.cc:290] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [CANDIDATE]: Term 1 election: Requested vote from peers 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:48.774727 22950 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b3c763a0a8d441284b698fe0949090d" candidate_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd"
I20251028 21:12:48.774753 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b3c763a0a8d441284b698fe0949090d" candidate_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8"
I20251028 21:12:48.774791 23101 raft_consensus.cc:3060] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.774791 22950 raft_consensus.cc:3060] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:48.775225 23101 raft_consensus.cc:2468] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b24afaeee5d4b2388b84d94ea28a92e in term 1.
I20251028 21:12:48.775326 22950 raft_consensus.cc:2468] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b24afaeee5d4b2388b84d94ea28a92e in term 1.
I20251028 21:12:48.775373 22992 leader_election.cc:304] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b24afaeee5d4b2388b84d94ea28a92e, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:48.775457 23154 raft_consensus.cc:2804] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:48.775563 23154 raft_consensus.cc:697] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 LEADER]: Becoming Leader. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Running, Role: LEADER
I20251028 21:12:48.775605 23154 consensus_queue.cc:237] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:48.776032 22847 catalog_manager.cc:5649] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2). New cstate: current_term: 1 leader_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:49.023942 23162 raft_consensus.cc:493] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:49.024040 23162 raft_consensus.cc:515] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.024209 23162 leader_election.cc:290] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405), 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217)
I20251028 21:12:49.024394 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6d1954a45164b32b866a241b9487e59" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" is_pre_election: true
I20251028 21:12:49.024417 22950 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6d1954a45164b32b866a241b9487e59" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd" is_pre_election: true
I20251028 21:12:49.024482 23026 raft_consensus.cc:2468] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 0.
I20251028 21:12:49.024494 22950 raft_consensus.cc:2468] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 0.
I20251028 21:12:49.024612 23066 leader_election.cc:304] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b24afaeee5d4b2388b84d94ea28a92e, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:49.024698 23162 raft_consensus.cc:2804] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:49.024737 23162 raft_consensus.cc:493] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:49.024751 23162 raft_consensus.cc:3060] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.025365 23162 raft_consensus.cc:515] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.025475 23162 leader_election.cc:290] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405), 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217)
I20251028 21:12:49.025612 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6d1954a45164b32b866a241b9487e59" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
I20251028 21:12:49.025638 22950 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6d1954a45164b32b866a241b9487e59" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd"
I20251028 21:12:49.025686 23026 raft_consensus.cc:3060] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.025697 22950 raft_consensus.cc:3060] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.026368 23026 raft_consensus.cc:2468] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 1.
I20251028 21:12:49.026368 22950 raft_consensus.cc:2468] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 1.
I20251028 21:12:49.026551 23066 leader_election.cc:304] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b24afaeee5d4b2388b84d94ea28a92e, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:49.026656 23162 raft_consensus.cc:2804] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:49.026696 23162 raft_consensus.cc:697] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Becoming Leader. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Running, Role: LEADER
I20251028 21:12:49.026751 23162 consensus_queue.cc:237] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.027318 22847 catalog_manager.cc:5649] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb4ab48d916944119f57263513c12fb8 (127.15.205.3). New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:49.105334 16180 test_util.cc:276] Using random seed: 2053435820
I20251028 21:12:49.110127 22847 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50566:
name: "dugtrio"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 21:12:49.110294 22847 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table dugtrio in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 21:12:49.111801 22938 tablet_service.cc:1505] Processing CreateTablet for tablet b939f312f1894117b1b7763c0d31bfea (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:49.111871 22937 tablet_service.cc:1505] Processing CreateTablet for tablet 4beefd9217ea4efc8fb21b2d94004b10 (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:49.111932 22938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b939f312f1894117b1b7763c0d31bfea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.112006 23090 tablet_service.cc:1505] Processing CreateTablet for tablet b939f312f1894117b1b7763c0d31bfea (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:49.112039 22940 tablet_service.cc:1505] Processing CreateTablet for tablet 1a987acc6a0046a1b638097e2fbcc163 (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:49.112004 22939 tablet_service.cc:1505] Processing CreateTablet for tablet 2471ae0ab78a4a4f8f925947ec69a49a (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:49.112104 23090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b939f312f1894117b1b7763c0d31bfea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.111994 22937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4beefd9217ea4efc8fb21b2d94004b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.112155 23013 tablet_service.cc:1505] Processing CreateTablet for tablet b939f312f1894117b1b7763c0d31bfea (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251028 21:12:49.112231 23013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b939f312f1894117b1b7763c0d31bfea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.113324 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2471ae0ab78a4a4f8f925947ec69a49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.113345 23016 tablet_service.cc:1505] Processing CreateTablet for tablet 2471ae0ab78a4a4f8f925947ec69a49a (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:49.113360 23088 tablet_service.cc:1505] Processing CreateTablet for tablet 2471ae0ab78a4a4f8f925947ec69a49a (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251028 21:12:49.113391 23148 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: Bootstrap starting.
I20251028 21:12:49.113411 23016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2471ae0ab78a4a4f8f925947ec69a49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.113442 23088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2471ae0ab78a4a4f8f925947ec69a49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.113879 23148 tablet_bootstrap.cc:654] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.114415 23014 tablet_service.cc:1505] Processing CreateTablet for tablet 4beefd9217ea4efc8fb21b2d94004b10 (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:49.114485 23014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4beefd9217ea4efc8fb21b2d94004b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.114565 23089 tablet_service.cc:1505] Processing CreateTablet for tablet 4beefd9217ea4efc8fb21b2d94004b10 (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251028 21:12:49.114624 23089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4beefd9217ea4efc8fb21b2d94004b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.115233 23147 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:49.115383 23091 tablet_service.cc:1505] Processing CreateTablet for tablet 1a987acc6a0046a1b638097e2fbcc163 (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:49.115442 23091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a987acc6a0046a1b638097e2fbcc163. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.115589 23147 tablet_bootstrap.cc:654] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.115828 23150 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: Bootstrap starting.
I20251028 21:12:49.116137 23015 tablet_service.cc:1505] Processing CreateTablet for tablet 1a987acc6a0046a1b638097e2fbcc163 (DEFAULT_TABLE table=dugtrio [id=e7857e99d97d425ebad5ece62d36376e]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251028 21:12:49.116199 23150 tablet_bootstrap.cc:654] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.116209 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a987acc6a0046a1b638097e2fbcc163. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.116648 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a987acc6a0046a1b638097e2fbcc163. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:49.117002 23148 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: No bootstrap required, opened a new log
I20251028 21:12:49.117041 23148 ts_tablet_manager.cc:1403] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251028 21:12:49.117178 23148 raft_consensus.cc:359] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.117238 23148 raft_consensus.cc:385] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.117257 23148 raft_consensus.cc:740] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.117311 23148 consensus_queue.cc:260] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.117393 23148 ts_tablet_manager.cc:1434] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.117439 23148 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: Bootstrap starting.
I20251028 21:12:49.117825 23148 tablet_bootstrap.cc:654] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.117918 23152 consensus_queue.cc:1048] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:49.118427 23148 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: No bootstrap required, opened a new log
I20251028 21:12:49.118475 23148 ts_tablet_manager.cc:1403] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:49.118587 23147 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: No bootstrap required, opened a new log
I20251028 21:12:49.118613 23148 raft_consensus.cc:359] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.118619 23147 ts_tablet_manager.cc:1403] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:12:49.118697 23148 raft_consensus.cc:385] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.118721 23148 raft_consensus.cc:740] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.118767 23148 consensus_queue.cc:260] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.118793 23150 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: No bootstrap required, opened a new log
I20251028 21:12:49.118817 23147 raft_consensus.cc:359] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.118852 23148 ts_tablet_manager.cc:1434] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.118901 23147 raft_consensus.cc:385] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.118827 23150 ts_tablet_manager.cc:1403] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 21:12:49.119082 23150 raft_consensus.cc:359] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.119141 23150 raft_consensus.cc:385] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.119167 23150 raft_consensus.cc:740] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.119202 23150 consensus_queue.cc:260] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.119303 23150 ts_tablet_manager.cc:1434] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.119347 23150 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: Bootstrap starting.
I20251028 21:12:49.119715 23150 tablet_bootstrap.cc:654] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.119745 23168 consensus_queue.cc:1048] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:49.118927 23148 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: Bootstrap starting.
I20251028 21:12:49.120528 23148 tablet_bootstrap.cc:654] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.120647 23150 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: No bootstrap required, opened a new log
I20251028 21:12:49.120697 23150 ts_tablet_manager.cc:1403] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:49.118927 23147 raft_consensus.cc:740] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.120855 23150 raft_consensus.cc:359] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.120889 23147 consensus_queue.cc:260] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.120931 23150 raft_consensus.cc:385] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.120954 23150 raft_consensus.cc:740] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.121009 23147 ts_tablet_manager.cc:1434] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251028 21:12:49.121037 23150 consensus_queue.cc:260] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.121059 23147 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:49.121130 23150 ts_tablet_manager.cc:1434] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:49.121189 23150 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: Bootstrap starting.
I20251028 21:12:49.121289 23148 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: No bootstrap required, opened a new log
I20251028 21:12:49.121335 23148 ts_tablet_manager.cc:1403] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:49.121481 23148 raft_consensus.cc:359] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.121512 23147 tablet_bootstrap.cc:654] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.121536 23148 raft_consensus.cc:385] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.121553 23148 raft_consensus.cc:740] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.121600 23150 tablet_bootstrap.cc:654] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.121589 23148 consensus_queue.cc:260] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.121685 23148 ts_tablet_manager.cc:1434] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.121733 23148 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: Bootstrap starting.
I20251028 21:12:49.122120 23148 tablet_bootstrap.cc:654] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.122588 23150 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: No bootstrap required, opened a new log
I20251028 21:12:49.122133 23147 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: No bootstrap required, opened a new log
I20251028 21:12:49.122700 23150 ts_tablet_manager.cc:1403] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:49.122726 23147 ts_tablet_manager.cc:1403] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:49.122838 23150 raft_consensus.cc:359] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.122841 23147 raft_consensus.cc:359] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.122889 23150 raft_consensus.cc:385] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.122982 23147 raft_consensus.cc:385] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.122996 23150 raft_consensus.cc:740] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.123005 23147 raft_consensus.cc:740] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.123054 23150 consensus_queue.cc:260] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.123068 23147 consensus_queue.cc:260] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.123138 23150 ts_tablet_manager.cc:1434] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.123152 23147 ts_tablet_manager.cc:1434] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.123155 23148 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: No bootstrap required, opened a new log
I20251028 21:12:49.123214 23148 ts_tablet_manager.cc:1403] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:49.123219 23147 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:49.123370 23148 raft_consensus.cc:359] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.123412 23148 raft_consensus.cc:385] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.123430 23148 raft_consensus.cc:740] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b24afaeee5d4b2388b84d94ea28a92e, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.123468 23148 consensus_queue.cc:260] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.123545 23148 ts_tablet_manager.cc:1434] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.123183 23150 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: Bootstrap starting.
I20251028 21:12:49.123761 23147 tablet_bootstrap.cc:654] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.123952 23150 tablet_bootstrap.cc:654] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.124286 23147 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: No bootstrap required, opened a new log
I20251028 21:12:49.124333 23147 ts_tablet_manager.cc:1403] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:49.124455 23147 raft_consensus.cc:359] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.124492 23150 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: No bootstrap required, opened a new log
I20251028 21:12:49.124487 23147 raft_consensus.cc:385] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.124564 23147 raft_consensus.cc:740] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.124567 23150 ts_tablet_manager.cc:1403] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:49.124619 23147 consensus_queue.cc:260] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.124709 23147 ts_tablet_manager.cc:1434] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.124763 23147 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:49.124744 23150 raft_consensus.cc:359] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.124802 23150 raft_consensus.cc:385] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.124825 23150 raft_consensus.cc:740] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.124881 23150 consensus_queue.cc:260] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.124969 23150 ts_tablet_manager.cc:1434] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.125128 23147 tablet_bootstrap.cc:654] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Neither blocks nor log segments found. Creating new log.
I20251028 21:12:49.125716 23147 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: No bootstrap required, opened a new log
I20251028 21:12:49.125753 23147 ts_tablet_manager.cc:1403] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251028 21:12:49.125878 23147 raft_consensus.cc:359] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.125967 23147 raft_consensus.cc:385] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 21:12:49.125988 23147 raft_consensus.cc:740] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:49.126114 23147 consensus_queue.cc:260] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.126245 23147 ts_tablet_manager.cc:1434] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:49.147722 23162 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:49.149108 23162 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:49.183337 23168 raft_consensus.cc:493] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:49.183418 23168 raft_consensus.cc:515] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.183578 23168 leader_election.cc:290] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:49.183723 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a987acc6a0046a1b638097e2fbcc163" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8" is_pre_election: true
I20251028 21:12:49.183723 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a987acc6a0046a1b638097e2fbcc163" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" is_pre_election: true
I20251028 21:12:49.183816 23101 raft_consensus.cc:2468] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:49.183816 23026 raft_consensus.cc:2468] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:49.183987 22916 leader_election.cc:304] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:49.184091 23168 raft_consensus.cc:2804] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:49.184124 23168 raft_consensus.cc:493] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:49.184141 23168 raft_consensus.cc:3060] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.184713 23168 raft_consensus.cc:515] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.184840 23168 leader_election.cc:290] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Requested vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:49.184971 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a987acc6a0046a1b638097e2fbcc163" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
I20251028 21:12:49.184986 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a987acc6a0046a1b638097e2fbcc163" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8"
I20251028 21:12:49.185029 23026 raft_consensus.cc:3060] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.185037 23101 raft_consensus.cc:3060] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.185552 23026 raft_consensus.cc:2468] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:49.185582 23101 raft_consensus.cc:2468] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:49.185703 22915 leader_election.cc:304] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, 7b24afaeee5d4b2388b84d94ea28a92e; no voters:
I20251028 21:12:49.185781 23168 raft_consensus.cc:2804] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:49.185854 23168 raft_consensus.cc:697] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Becoming Leader. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Running, Role: LEADER
I20251028 21:12:49.185913 23168 consensus_queue.cc:237] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.186393 22846 catalog_manager.cc:5649] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a25731407814b3bb1a4125abce11bbd (127.15.205.1). New cstate: current_term: 1 leader_uuid: "3a25731407814b3bb1a4125abce11bbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:49.203073 23154 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:49.204193 23204 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 21:12:49.247048 23168 raft_consensus.cc:493] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:49.247149 23168 raft_consensus.cc:515] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.247306 23168 leader_election.cc:290] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:49.247443 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4beefd9217ea4efc8fb21b2d94004b10" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" is_pre_election: true
I20251028 21:12:49.247463 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4beefd9217ea4efc8fb21b2d94004b10" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8" is_pre_election: true
I20251028 21:12:49.247531 23026 raft_consensus.cc:2468] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:49.247537 23101 raft_consensus.cc:2468] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:49.247673 22915 leader_election.cc:304] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, 7b24afaeee5d4b2388b84d94ea28a92e; no voters:
I20251028 21:12:49.247766 23168 raft_consensus.cc:2804] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:49.247828 23168 raft_consensus.cc:493] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:49.247851 23168 raft_consensus.cc:3060] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.248478 23168 raft_consensus.cc:515] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.248615 23168 leader_election.cc:290] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Requested vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:49.248747 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4beefd9217ea4efc8fb21b2d94004b10" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
I20251028 21:12:49.248756 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4beefd9217ea4efc8fb21b2d94004b10" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8"
I20251028 21:12:49.248837 23101 raft_consensus.cc:3060] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.248826 23026 raft_consensus.cc:3060] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.249506 23026 raft_consensus.cc:2468] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:49.249506 23101 raft_consensus.cc:2468] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:49.249679 22915 leader_election.cc:304] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, 7b24afaeee5d4b2388b84d94ea28a92e; no voters:
I20251028 21:12:49.249763 23168 raft_consensus.cc:2804] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:49.249814 23168 raft_consensus.cc:697] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Becoming Leader. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Running, Role: LEADER
I20251028 21:12:49.249893 23168 consensus_queue.cc:237] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.250424 22847 catalog_manager.cc:5649] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a25731407814b3bb1a4125abce11bbd (127.15.205.1). New cstate: current_term: 1 leader_uuid: "3a25731407814b3bb1a4125abce11bbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:49.409132 23168 raft_consensus.cc:493] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:49.409227 23168 raft_consensus.cc:515] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.409358 23168 leader_election.cc:290] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:49.409548 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2471ae0ab78a4a4f8f925947ec69a49a" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" is_pre_election: true
I20251028 21:12:49.409554 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2471ae0ab78a4a4f8f925947ec69a49a" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8" is_pre_election: true
I20251028 21:12:49.409637 23026 raft_consensus.cc:2468] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:49.409637 23101 raft_consensus.cc:2468] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:49.409817 22915 leader_election.cc:304] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, 7b24afaeee5d4b2388b84d94ea28a92e; no voters:
I20251028 21:12:49.409931 23168 raft_consensus.cc:2804] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:49.409976 23168 raft_consensus.cc:493] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:49.409998 23168 raft_consensus.cc:3060] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.410593 23168 raft_consensus.cc:515] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.410699 23168 leader_election.cc:290] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Requested vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:49.410805 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2471ae0ab78a4a4f8f925947ec69a49a" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8"
I20251028 21:12:49.410815 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2471ae0ab78a4a4f8f925947ec69a49a" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
I20251028 21:12:49.410878 23101 raft_consensus.cc:3060] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.410887 23026 raft_consensus.cc:3060] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.411396 23101 raft_consensus.cc:2468] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:49.411417 23026 raft_consensus.cc:2468] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:49.411556 22916 leader_election.cc:304] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:49.411634 23168 raft_consensus.cc:2804] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:49.411686 23168 raft_consensus.cc:697] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Becoming Leader. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Running, Role: LEADER
I20251028 21:12:49.411731 23168 consensus_queue.cc:237] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.412324 22847 catalog_manager.cc:5649] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a25731407814b3bb1a4125abce11bbd (127.15.205.1). New cstate: current_term: 1 leader_uuid: "3a25731407814b3bb1a4125abce11bbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:49.460314 23168 raft_consensus.cc:493] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 21:12:49.460422 23168 raft_consensus.cc:515] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.460585 23168 leader_election.cc:290] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:49.460810 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b939f312f1894117b1b7763c0d31bfea" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8" is_pre_election: true
I20251028 21:12:49.460810 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b939f312f1894117b1b7763c0d31bfea" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" is_pre_election: true
I20251028 21:12:49.460906 23101 raft_consensus.cc:2468] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:49.460911 23026 raft_consensus.cc:2468] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 0.
I20251028 21:12:49.461059 22915 leader_election.cc:304] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, 7b24afaeee5d4b2388b84d94ea28a92e; no voters:
I20251028 21:12:49.461161 23168 raft_consensus.cc:2804] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 21:12:49.461231 23168 raft_consensus.cc:493] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 21:12:49.461253 23168 raft_consensus.cc:3060] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.461830 23168 raft_consensus.cc:515] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.462005 23168 leader_election.cc:290] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Requested vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:49.462087 23026 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b939f312f1894117b1b7763c0d31bfea" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
I20251028 21:12:49.462075 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b939f312f1894117b1b7763c0d31bfea" candidate_uuid: "3a25731407814b3bb1a4125abce11bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8"
I20251028 21:12:49.462142 23026 raft_consensus.cc:3060] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.462163 23101 raft_consensus.cc:3060] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [term 0 FOLLOWER]: Advancing to term 1
I20251028 21:12:49.462744 23026 raft_consensus.cc:2468] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:49.462744 23101 raft_consensus.cc:2468] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25731407814b3bb1a4125abce11bbd in term 1.
I20251028 21:12:49.462924 22916 leader_election.cc:304] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:49.463009 23168 raft_consensus.cc:2804] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader election won for term 1
I20251028 21:12:49.463055 23168 raft_consensus.cc:697] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Becoming Leader. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Running, Role: LEADER
I20251028 21:12:49.463107 23168 consensus_queue.cc:237] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:49.463642 22847 catalog_manager.cc:5649] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a25731407814b3bb1a4125abce11bbd (127.15.205.1). New cstate: current_term: 1 leader_uuid: "3a25731407814b3bb1a4125abce11bbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:49.464488 16180 tablet_server.cc:178] TabletServer@127.15.205.1:0 shutting down...
I20251028 21:12:49.467680 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:49.467854 16180 tablet_replica.cc:333] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: stopping tablet replica
I20251028 21:12:49.467916 16180 raft_consensus.cc:2243] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:49.467970 16180 pending_rounds.cc:70] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Trying to abort 1 pending ops.
I20251028 21:12:49.467995 16180 pending_rounds.cc:77] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865729785921536 op_type: NO_OP noop_request { }
I20251028 21:12:49.468037 16180 raft_consensus.cc:2889] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:49.468070 16180 raft_consensus.cc:2272] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:49.468313 16180 tablet_replica.cc:333] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: stopping tablet replica
I20251028 21:12:49.468360 16180 raft_consensus.cc:2243] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:49.468405 16180 pending_rounds.cc:70] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Trying to abort 1 pending ops.
I20251028 21:12:49.468429 16180 pending_rounds.cc:77] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865730710851584 op_type: NO_OP noop_request { }
I20251028 21:12:49.468468 16180 raft_consensus.cc:2889] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:49.468499 16180 raft_consensus.cc:2272] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:49.468739 16180 tablet_replica.cc:333] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: stopping tablet replica
I20251028 21:12:49.468788 16180 raft_consensus.cc:2243] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:49.468827 16180 raft_consensus.cc:2272] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:49.469023 16180 tablet_replica.cc:333] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: stopping tablet replica
I20251028 21:12:49.469060 16180 raft_consensus.cc:2243] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:49.469112 16180 raft_consensus.cc:2272] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:49.469277 16180 tablet_replica.cc:333] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: stopping tablet replica
I20251028 21:12:49.469316 16180 raft_consensus.cc:2243] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:49.469352 16180 raft_consensus.cc:2272] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:49.469446 16180 tablet_replica.cc:333] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: stopping tablet replica
I20251028 21:12:49.469480 16180 raft_consensus.cc:2243] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:49.469530 16180 pending_rounds.cc:70] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Trying to abort 1 pending ops.
I20251028 21:12:49.469573 16180 pending_rounds.cc:77] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865730921361408 op_type: NO_OP noop_request { }
I20251028 21:12:49.469610 16180 raft_consensus.cc:2889] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:49.469640 16180 raft_consensus.cc:2272] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:49.469847 16180 tablet_replica.cc:333] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: stopping tablet replica
I20251028 21:12:49.469894 16180 raft_consensus.cc:2243] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:49.469929 16180 raft_consensus.cc:2272] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:49.470084 16180 tablet_replica.cc:333] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: stopping tablet replica
I20251028 21:12:49.470120 16180 raft_consensus.cc:2243] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:49.470160 16180 pending_rounds.cc:70] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Trying to abort 1 pending ops.
I20251028 21:12:49.470181 16180 pending_rounds.cc:77] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865730048057344 op_type: NO_OP noop_request { }
I20251028 21:12:49.470216 16180 raft_consensus.cc:2889] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:49.470247 16180 raft_consensus.cc:2272] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:49.482331 16180 tablet_server.cc:195] TabletServer@127.15.205.1:0 shutdown complete.
W20251028 21:12:49.595175 23067 consensus_peers.cc:597] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 -> Peer 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): Couldn't send request to peer 3a25731407814b3bb1a4125abce11bbd. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:41405: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:49.599880 23162 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 21:12:49.696427 23067 consensus_peers.cc:597] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 -> Peer 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): Couldn't send request to peer 3a25731407814b3bb1a4125abce11bbd. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:41405: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251028 21:12:49.725402 22992 consensus_peers.cc:597] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e -> Peer 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): Couldn't send request to peer 3a25731407814b3bb1a4125abce11bbd. Status: Network error: Client connection negotiation failed: client connection to 127.15.205.1:41405: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:50.083120 23212 consensus_queue.cc:579] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Leader has been unable to successfully communicate with peer 3a25731407814b3bb1a4125abce11bbd for more than 1 seconds (1.056s)
I20251028 21:12:50.285151 23158 consensus_queue.cc:579] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Leader has been unable to successfully communicate with peer 3a25731407814b3bb1a4125abce11bbd for more than 1 seconds (1.135s)
I20251028 21:12:50.486649 22846 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251028 21:12:50.486740 22846 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251028 21:12:50.487728 22846 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: Sending DeleteTablet for 3 replicas of tablet b939f312f1894117b1b7763c0d31bfea
I20251028 21:12:50.487839 22846 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: Sending DeleteTablet for 3 replicas of tablet 2471ae0ab78a4a4f8f925947ec69a49a
I20251028 21:12:50.487887 22846 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: Sending DeleteTablet for 3 replicas of tablet 4beefd9217ea4efc8fb21b2d94004b10
I20251028 21:12:50.487953 22846 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: Sending DeleteTablet for 3 replicas of tablet 1a987acc6a0046a1b638097e2fbcc163
I20251028 21:12:50.488147 23016 tablet_service.cc:1552] Processing DeleteTablet for tablet 4beefd9217ea4efc8fb21b2d94004b10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:54498
I20251028 21:12:50.488188 23013 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a987acc6a0046a1b638097e2fbcc163 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:54498
I20251028 21:12:50.488225 23015 tablet_service.cc:1552] Processing DeleteTablet for tablet b939f312f1894117b1b7763c0d31bfea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:54498
I20251028 21:12:50.488160 23014 tablet_service.cc:1552] Processing DeleteTablet for tablet 2471ae0ab78a4a4f8f925947ec69a49a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:54498
I20251028 21:12:50.488327 23091 tablet_service.cc:1552] Processing DeleteTablet for tablet b939f312f1894117b1b7763c0d31bfea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:34750
I20251028 21:12:50.488344 23224 tablet_replica.cc:333] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: stopping tablet replica
I20251028 21:12:50.488392 23224 raft_consensus.cc:2243] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.488420 23224 raft_consensus.cc:2272] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.488453 23089 tablet_service.cc:1552] Processing DeleteTablet for tablet 2471ae0ab78a4a4f8f925947ec69a49a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:34750
I20251028 21:12:50.488487 23226 tablet_replica.cc:333] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: stopping tablet replica
I20251028 21:12:50.488515 23088 tablet_service.cc:1552] Processing DeleteTablet for tablet 4beefd9217ea4efc8fb21b2d94004b10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:34750
I20251028 21:12:50.488535 23226 raft_consensus.cc:2243] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.488572 23224 ts_tablet_manager.cc:1916] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.488564 23226 raft_consensus.cc:2272] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.488566 23090 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a987acc6a0046a1b638097e2fbcc163 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:34750
I20251028 21:12:50.488775 23226 ts_tablet_manager.cc:1916] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.488830 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 21:12:50.489703 22834 catalog_manager.cc:4712] TS 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a987acc6a0046a1b638097e2fbcc163: Network error: Client connection negotiation failed: client connection to 127.15.205.1:41405: connect: Connection refused (error 111)
W20251028 21:12:50.490043 23227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:50.490217 23226 ts_tablet_manager.cc:1929] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251028 21:12:50.490280 23230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:50.490327 23224 ts_tablet_manager.cc:1929] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 21:12:50.490346 23226 log.cc:1199] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/b939f312f1894117b1b7763c0d31bfea
I20251028 21:12:50.490458 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:50.490564 23224 log.cc:1199] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/4beefd9217ea4efc8fb21b2d94004b10
W20251028 21:12:50.490825 23228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:50.490899 23226 ts_tablet_manager.cc:1950] T b939f312f1894117b1b7763c0d31bfea P bb4ab48d916944119f57263513c12fb8: Deleting consensus metadata
I20251028 21:12:50.490974 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251028 21:12:50.490972 23224 ts_tablet_manager.cc:1950] T 4beefd9217ea4efc8fb21b2d94004b10 P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting consensus metadata
W20251028 21:12:50.491030 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:50.491055 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685970491054 us; error 0 us; skew 500 ppm
I20251028 21:12:50.491165 23226 tablet_replica.cc:333] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: stopping tablet replica
I20251028 21:12:50.491237 22834 catalog_manager.cc:4985] TS bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395): tablet b939f312f1894117b1b7763c0d31bfea (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.491253 23224 tablet_replica.cc:333] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: stopping tablet replica
I20251028 21:12:50.491379 23224 raft_consensus.cc:2243] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.491425 23224 raft_consensus.cc:2272] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.491576 23224 ts_tablet_manager.cc:1916] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.491600 22833 catalog_manager.cc:4985] TS 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217): tablet 4beefd9217ea4efc8fb21b2d94004b10 (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.491674 16180 webserver.cc:492] Webserver started at http://127.15.205.4:36745/ using document root <none> and password file <none>
I20251028 21:12:50.491654 23226 raft_consensus.cc:2243] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.491806 23226 raft_consensus.cc:2272] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.491832 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:50.491868 16180 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 21:12:50.491914 16180 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 21:12:50.491940 23226 ts_tablet_manager.cc:1916] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.492093 16180 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-3-root/instance:
uuid: "9c8bcb21e08e4d9fb3e681731549601b"
format_stamp: "Formatted at 2025-10-28 21:12:50 on dist-test-slave-zmkx"
I20251028 21:12:50.492816 16180 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:50.492892 23224 ts_tablet_manager.cc:1929] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 21:12:50.492946 23224 log.cc:1199] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/1a987acc6a0046a1b638097e2fbcc163
I20251028 21:12:50.493100 23224 ts_tablet_manager.cc:1950] T 1a987acc6a0046a1b638097e2fbcc163 P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting consensus metadata
I20251028 21:12:50.493113 23226 ts_tablet_manager.cc:1929] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 21:12:50.493148 23226 log.cc:1199] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/2471ae0ab78a4a4f8f925947ec69a49a
I20251028 21:12:50.493228 23235 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.493283 23226 ts_tablet_manager.cc:1950] T 2471ae0ab78a4a4f8f925947ec69a49a P bb4ab48d916944119f57263513c12fb8: Deleting consensus metadata
I20251028 21:12:50.493326 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.493361 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-3-root
uuid: "9c8bcb21e08e4d9fb3e681731549601b"
format_stamp: "Formatted at 2025-10-28 21:12:50 on dist-test-slave-zmkx"
I20251028 21:12:50.493393 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:50.493574 23226 tablet_replica.cc:333] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: stopping tablet replica
I20251028 21:12:50.493561 22833 catalog_manager.cc:4985] TS 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217): tablet 1a987acc6a0046a1b638097e2fbcc163 (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.493646 23226 raft_consensus.cc:2243] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.493680 23226 raft_consensus.cc:2272] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.493574 23224 tablet_replica.cc:333] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: stopping tablet replica
I20251028 21:12:50.493749 23224 raft_consensus.cc:2243] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.493798 23226 ts_tablet_manager.cc:1916] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.493825 23224 raft_consensus.cc:2272] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.493986 23224 ts_tablet_manager.cc:1916] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.494694 22834 catalog_manager.cc:4985] TS bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395): tablet 2471ae0ab78a4a4f8f925947ec69a49a (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.494805 23226 ts_tablet_manager.cc:1929] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 21:12:50.494850 23226 log.cc:1199] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/4beefd9217ea4efc8fb21b2d94004b10
I20251028 21:12:50.495007 23226 ts_tablet_manager.cc:1950] T 4beefd9217ea4efc8fb21b2d94004b10 P bb4ab48d916944119f57263513c12fb8: Deleting consensus metadata
I20251028 21:12:50.495010 23224 ts_tablet_manager.cc:1929] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 21:12:50.495071 23224 log.cc:1199] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/b939f312f1894117b1b7763c0d31bfea
I20251028 21:12:50.495165 23226 tablet_replica.cc:333] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: stopping tablet replica
I20251028 21:12:50.495205 23226 raft_consensus.cc:2243] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.495209 23224 ts_tablet_manager.cc:1950] T b939f312f1894117b1b7763c0d31bfea P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting consensus metadata
I20251028 21:12:50.495247 22834 catalog_manager.cc:4985] TS bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395): tablet 4beefd9217ea4efc8fb21b2d94004b10 (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.495252 23226 raft_consensus.cc:2272] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.495432 23226 ts_tablet_manager.cc:1916] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.495568 23224 tablet_replica.cc:333] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: stopping tablet replica
I20251028 21:12:50.495604 23224 raft_consensus.cc:2243] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.495627 23224 raft_consensus.cc:2272] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.495658 22833 catalog_manager.cc:4985] TS 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217): tablet b939f312f1894117b1b7763c0d31bfea (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.495713 23224 ts_tablet_manager.cc:1916] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.496402 23226 ts_tablet_manager.cc:1929] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 21:12:50.496448 23226 log.cc:1199] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/1a987acc6a0046a1b638097e2fbcc163
I20251028 21:12:50.496630 23226 ts_tablet_manager.cc:1950] T 1a987acc6a0046a1b638097e2fbcc163 P bb4ab48d916944119f57263513c12fb8: Deleting consensus metadata
I20251028 21:12:50.496680 23224 ts_tablet_manager.cc:1929] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 21:12:50.496727 23224 log.cc:1199] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/2471ae0ab78a4a4f8f925947ec69a49a
I20251028 21:12:50.496858 23224 ts_tablet_manager.cc:1950] T 2471ae0ab78a4a4f8f925947ec69a49a P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting consensus metadata
I20251028 21:12:50.496881 22834 catalog_manager.cc:4985] TS bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395): tablet 1a987acc6a0046a1b638097e2fbcc163 (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.497169 22833 catalog_manager.cc:4985] TS 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217): tablet 2471ae0ab78a4a4f8f925947ec69a49a (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.520805 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:50.520951 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:50.521071 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:50.521243 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 21:12:50.521270 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.521299 16180 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 21:12:50.521327 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.523815 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.4:42181
I20251028 21:12:50.524276 16180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 21:12:50.524384 23298 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.4:42181 every 8 connection(s)
W20251028 21:12:50.525319 23301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:50.525334 23302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 21:12:50.525372 23305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 21:12:50.525578 16180 server_base.cc:1047] running on GCE node
I20251028 21:12:50.525667 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251028 21:12:50.525696 16180 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251028 21:12:50.525718 16180 hybrid_clock.cc:648] HybridClock initialized: now 1761685970525718 us; error 0 us; skew 500 ppm
I20251028 21:12:50.526309 16180 webserver.cc:492] Webserver started at http://127.15.205.1:38883/ using document root <none> and password file <none>
I20251028 21:12:50.526386 16180 fs_manager.cc:362] Metadata directory not provided
I20251028 21:12:50.526422 16180 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251028 21:12:50.527186 16180 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:50.527617 23299 heartbeater.cc:344] Connected to a master server at 127.15.205.62:33985
I20251028 21:12:50.527683 23299 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:50.527781 23299 heartbeater.cc:507] Master 127.15.205.62:33985 requested a full tablet report, sending...
I20251028 21:12:50.528021 22846 ts_manager.cc:194] Registered new tserver with Master: 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4:42181)
I20251028 21:12:50.528122 23310 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.528257 16180 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251028 21:12:50.528306 16180 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root
uuid: "3a25731407814b3bb1a4125abce11bbd"
format_stamp: "Formatted at 2025-10-28 21:12:47 on dist-test-slave-zmkx"
I20251028 21:12:50.528352 16180 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 21:12:50.528479 22846 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50590
I20251028 21:12:50.540709 16180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 21:12:50.540839 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 21:12:50.540931 16180 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 21:12:50.541136 23317 ts_tablet_manager.cc:542] Loading tablet metadata (0/8 complete)
W20251028 21:12:50.542318 22896 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 3a25731407814b3bb1a4125abce11bbd not available for placement
I20251028 21:12:50.543445 16180 ts_tablet_manager.cc:585] Loaded tablet metadata (8 total tablets, 8 live tablets)
I20251028 21:12:50.543500 16180 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20251028 21:12:50.543545 16180 ts_tablet_manager.cc:600] Registering tablets (0/8 complete)
I20251028 21:12:50.543982 23317 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:50.544943 23317 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251028 21:12:50.545185 16180 ts_tablet_manager.cc:616] Registered 8 tablets
I20251028 21:12:50.545190 23317 tablet_bootstrap.cc:492] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap complete.
I20251028 21:12:50.545244 16180 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20251028 21:12:50.545325 23317 ts_tablet_manager.cc:1403] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:50.545449 23317 raft_consensus.cc:359] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.545533 23317 raft_consensus.cc:740] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:50.545603 23317 consensus_queue.cc:260] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.545761 23317 ts_tablet_manager.cc:1434] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.545822 23317 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:50.546802 23317 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251028 21:12:50.547024 23317 tablet_bootstrap.cc:492] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Bootstrap complete.
I20251028 21:12:50.547132 23317 ts_tablet_manager.cc:1403] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:50.547240 23317 raft_consensus.cc:359] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.547329 23317 raft_consensus.cc:740] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:50.547374 23317 consensus_queue.cc:260] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.547458 23317 ts_tablet_manager.cc:1434] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.547502 23317 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:50.548386 16180 rpc_server.cc:307] RPC server started. Bound to: 127.15.205.1:41405
I20251028 21:12:50.548429 23378 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.205.1:41405 every 8 connection(s)
I20251028 21:12:50.548519 23317 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:50.548743 23317 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Bootstrap complete.
I20251028 21:12:50.548846 23317 ts_tablet_manager.cc:1403] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:50.548961 23317 raft_consensus.cc:359] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.549036 23317 raft_consensus.cc:740] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:50.549093 23317 consensus_queue.cc:260] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.549228 23317 ts_tablet_manager.cc:1434] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.549280 23317 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:50.550236 23317 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251028 21:12:50.550443 23317 tablet_bootstrap.cc:492] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Bootstrap complete.
I20251028 21:12:50.550545 23317 ts_tablet_manager.cc:1403] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:50.550684 23317 raft_consensus.cc:359] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.550751 23317 raft_consensus.cc:740] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:50.550805 23317 consensus_queue.cc:260] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.550906 23317 ts_tablet_manager.cc:1434] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:50.550978 23317 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:50.551795 23379 heartbeater.cc:344] Connected to a master server at 127.15.205.62:33985
I20251028 21:12:50.551860 23379 heartbeater.cc:461] Registering TS with master...
I20251028 21:12:50.551957 23317 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:50.551966 23379 heartbeater.cc:507] Master 127.15.205.62:33985 requested a full tablet report, sending...
I20251028 21:12:50.552119 23317 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap complete.
I20251028 21:12:50.552212 23317 ts_tablet_manager.cc:1403] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:50.552296 22846 ts_manager.cc:194] Re-registered known tserver with Master: 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405)
I20251028 21:12:50.552330 23317 raft_consensus.cc:359] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.552376 23317 raft_consensus.cc:740] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:50.552419 23317 consensus_queue.cc:260] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.552488 23317 ts_tablet_manager.cc:1434] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.552531 23317 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:50.552757 22846 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50604
I20251028 21:12:50.553117 23379 heartbeater.cc:499] Master 127.15.205.62:33985 was elected leader, sending a full tablet report...
I20251028 21:12:50.553467 23317 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251028 21:12:50.553653 23317 tablet_bootstrap.cc:492] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap complete.
I20251028 21:12:50.553730 23317 ts_tablet_manager.cc:1403] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:50.553851 23317 raft_consensus.cc:359] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.553895 23317 raft_consensus.cc:740] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:50.553931 23317 consensus_queue.cc:260] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.554008 23317 ts_tablet_manager.cc:1434] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.554059 23317 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:50.555135 23317 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:50.555317 23317 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap complete.
I20251028 21:12:50.555388 23317 ts_tablet_manager.cc:1403] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251028 21:12:50.555496 23317 raft_consensus.cc:359] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.555534 23317 raft_consensus.cc:740] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:50.555570 23317 consensus_queue.cc:260] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.555644 23317 ts_tablet_manager.cc:1434] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.555692 23317 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap starting.
I20251028 21:12:50.556716 23317 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:50.556875 23317 tablet_bootstrap.cc:492] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Bootstrap complete.
I20251028 21:12:50.556954 23317 ts_tablet_manager.cc:1403] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:50.557070 23317 raft_consensus.cc:359] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:50.557109 23317 raft_consensus.cc:740] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25731407814b3bb1a4125abce11bbd, State: Initialized, Role: FOLLOWER
I20251028 21:12:50.557173 23317 consensus_queue.cc:260] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:50.557267 23317 ts_tablet_manager.cc:1434] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:50.582608 23342 tablet_service.cc:1552] Processing DeleteTablet for tablet 1a987acc6a0046a1b638097e2fbcc163 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:39452
I20251028 21:12:50.582788 23390 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:50.582839 23390 raft_consensus.cc:2243] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.582867 23390 pending_rounds.cc:70] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Trying to abort 1 pending ops.
I20251028 21:12:50.582885 23390 pending_rounds.cc:77] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865729785921536 op_type: NO_OP noop_request { }
I20251028 21:12:50.582919 23390 raft_consensus.cc:2889] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:50.582939 23390 raft_consensus.cc:2272] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.583145 23390 ts_tablet_manager.cc:1916] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.584596 23390 ts_tablet_manager.cc:1929] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 21:12:50.584645 23390 log.cc:1199] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/1a987acc6a0046a1b638097e2fbcc163
I20251028 21:12:50.584810 23390 ts_tablet_manager.cc:1950] T 1a987acc6a0046a1b638097e2fbcc163 P 3a25731407814b3bb1a4125abce11bbd: Deleting consensus metadata
I20251028 21:12:50.585057 22834 catalog_manager.cc:4985] TS 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): tablet 1a987acc6a0046a1b638097e2fbcc163 (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.585203 23342 tablet_service.cc:1552] Processing DeleteTablet for tablet 4beefd9217ea4efc8fb21b2d94004b10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:39452
I20251028 21:12:50.585281 23390 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:50.585322 23390 raft_consensus.cc:2243] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.585346 23390 pending_rounds.cc:70] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Trying to abort 1 pending ops.
I20251028 21:12:50.585366 23390 pending_rounds.cc:77] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865730048057344 op_type: NO_OP noop_request { }
I20251028 21:12:50.585397 23390 raft_consensus.cc:2889] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:50.585417 23390 raft_consensus.cc:2272] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.585582 23390 ts_tablet_manager.cc:1916] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.586457 23390 ts_tablet_manager.cc:1929] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 21:12:50.586500 23390 log.cc:1199] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/4beefd9217ea4efc8fb21b2d94004b10
I20251028 21:12:50.586630 23390 ts_tablet_manager.cc:1950] T 4beefd9217ea4efc8fb21b2d94004b10 P 3a25731407814b3bb1a4125abce11bbd: Deleting consensus metadata
I20251028 21:12:50.586885 22834 catalog_manager.cc:4985] TS 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): tablet 4beefd9217ea4efc8fb21b2d94004b10 (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.608218 23342 tablet_service.cc:1552] Processing DeleteTablet for tablet 2471ae0ab78a4a4f8f925947ec69a49a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:39452
I20251028 21:12:50.608318 23390 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:50.608381 23390 raft_consensus.cc:2243] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.608426 23390 pending_rounds.cc:70] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Trying to abort 1 pending ops.
I20251028 21:12:50.608457 23390 pending_rounds.cc:77] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865730710851584 op_type: NO_OP noop_request { }
I20251028 21:12:50.608491 23390 raft_consensus.cc:2889] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:50.608510 23390 raft_consensus.cc:2272] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.608698 23390 ts_tablet_manager.cc:1916] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.609882 23390 ts_tablet_manager.cc:1929] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 21:12:50.609927 23390 log.cc:1199] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/2471ae0ab78a4a4f8f925947ec69a49a
I20251028 21:12:50.610087 23390 ts_tablet_manager.cc:1950] T 2471ae0ab78a4a4f8f925947ec69a49a P 3a25731407814b3bb1a4125abce11bbd: Deleting consensus metadata
I20251028 21:12:50.610344 22834 catalog_manager.cc:4985] TS 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): tablet 2471ae0ab78a4a4f8f925947ec69a49a (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.615671 23342 tablet_service.cc:1552] Processing DeleteTablet for tablet b939f312f1894117b1b7763c0d31bfea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 21:12:50 UTC) from {username='slave'} at 127.0.0.1:39452
I20251028 21:12:50.615753 23390 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:50.615792 23390 raft_consensus.cc:2243] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:50.615819 23390 pending_rounds.cc:70] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Trying to abort 1 pending ops.
I20251028 21:12:50.615835 23390 pending_rounds.cc:77] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215865730921361408 op_type: NO_OP noop_request { }
I20251028 21:12:50.615862 23390 raft_consensus.cc:2889] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 21:12:50.615877 23390 raft_consensus.cc:2272] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:50.616048 23390 ts_tablet_manager.cc:1916] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 21:12:50.616992 23390 ts_tablet_manager.cc:1929] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 21:12:50.617034 23390 log.cc:1199] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/b939f312f1894117b1b7763c0d31bfea
I20251028 21:12:50.617174 23390 ts_tablet_manager.cc:1950] T b939f312f1894117b1b7763c0d31bfea P 3a25731407814b3bb1a4125abce11bbd: Deleting consensus metadata
I20251028 21:12:50.617393 22834 catalog_manager.cc:4985] TS 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): tablet b939f312f1894117b1b7763c0d31bfea (table dugtrio [id=e7857e99d97d425ebad5ece62d36376e]) successfully deleted
I20251028 21:12:50.620432 23216 raft_consensus.cc:493] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a25731407814b3bb1a4125abce11bbd)
I20251028 21:12:50.620474 23216 raft_consensus.cc:515] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:50.620589 23216 leader_election.cc:290] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395), 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405)
I20251028 21:12:50.620658 23162 raft_consensus.cc:493] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a25731407814b3bb1a4125abce11bbd)
I20251028 21:12:50.620723 23162 raft_consensus.cc:515] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:50.620725 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8" is_pre_election: true
I20251028 21:12:50.620790 23101 raft_consensus.cc:2468] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b24afaeee5d4b2388b84d94ea28a92e in term 1.
I20251028 21:12:50.620833 23162 leader_election.cc:290] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405)
I20251028 21:12:50.621008 22992 leader_election.cc:304] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b24afaeee5d4b2388b84d94ea28a92e, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:50.621043 23025 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" is_pre_election: true
I20251028 21:12:50.621104 23216 raft_consensus.cc:2804] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 21:12:50.621043 23352 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd" is_pre_election: true
I20251028 21:12:50.621137 23025 raft_consensus.cc:2468] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 1.
I20251028 21:12:50.621145 23352 raft_consensus.cc:2468] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 1.
I20251028 21:12:50.621193 23216 raft_consensus.cc:493] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a25731407814b3bb1a4125abce11bbd)
I20251028 21:12:50.621238 23216 raft_consensus.cc:3060] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:50.621310 23067 leader_election.cc:304] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, bb4ab48d916944119f57263513c12fb8; no voters:
I20251028 21:12:50.621522 23162 raft_consensus.cc:2804] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 21:12:50.621577 23162 raft_consensus.cc:493] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a25731407814b3bb1a4125abce11bbd)
I20251028 21:12:50.621606 23162 raft_consensus.cc:3060] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:50.621950 23216 raft_consensus.cc:515] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:50.622082 23216 leader_election.cc:290] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [CANDIDATE]: Term 2 election: Requested vote from peers bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395), 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405)
I20251028 21:12:50.622287 23162 raft_consensus.cc:515] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:50.622421 23162 leader_election.cc:290] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217), 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405)
I20251028 21:12:50.622512 23025 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7b24afaeee5d4b2388b84d94ea28a92e"
I20251028 21:12:50.622574 23025 raft_consensus.cc:2393] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bb4ab48d916944119f57263513c12fb8 in current term 2: Already voted for candidate 7b24afaeee5d4b2388b84d94ea28a92e in this term.
I20251028 21:12:50.622594 23352 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "bb4ab48d916944119f57263513c12fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd"
I20251028 21:12:50.622653 23352 raft_consensus.cc:3060] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Advancing to term 2
I20251028 21:12:50.622747 23101 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bb4ab48d916944119f57263513c12fb8"
I20251028 21:12:50.622805 23101 raft_consensus.cc:2393] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7b24afaeee5d4b2388b84d94ea28a92e in current term 2: Already voted for candidate bb4ab48d916944119f57263513c12fb8 in this term.
I20251028 21:12:50.623337 23352 raft_consensus.cc:2468] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb4ab48d916944119f57263513c12fb8 in term 2.
I20251028 21:12:50.623479 23067 leader_election.cc:304] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a25731407814b3bb1a4125abce11bbd, bb4ab48d916944119f57263513c12fb8; no voters: 7b24afaeee5d4b2388b84d94ea28a92e
I20251028 21:12:50.623560 23162 raft_consensus.cc:2804] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20251028 21:12:50.623590 23352 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd" is_pre_election: true
I20251028 21:12:50.623613 23162 raft_consensus.cc:697] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 2 LEADER]: Becoming Leader. State: Replica: bb4ab48d916944119f57263513c12fb8, State: Running, Role: LEADER
I20251028 21:12:50.623624 23351 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f9a28ac41a04f3990922179532c2ca9" candidate_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a25731407814b3bb1a4125abce11bbd"
I20251028 21:12:50.623656 23352 raft_consensus.cc:2393] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7b24afaeee5d4b2388b84d94ea28a92e in current term 2: Already voted for candidate bb4ab48d916944119f57263513c12fb8 in this term.
I20251028 21:12:50.623680 23162 consensus_queue.cc:237] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } }
I20251028 21:12:50.623818 22992 leader_election.cc:304] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b24afaeee5d4b2388b84d94ea28a92e; no voters: 3a25731407814b3bb1a4125abce11bbd, bb4ab48d916944119f57263513c12fb8
I20251028 21:12:50.623932 23216 raft_consensus.cc:2749] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251028 21:12:50.624075 22846 catalog_manager.cc:5649] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 reported cstate change: term changed from 1 to 2, leader changed from 3a25731407814b3bb1a4125abce11bbd (127.15.205.1) to bb4ab48d916944119f57263513c12fb8 (127.15.205.3). New cstate: current_term: 2 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:50.646018 23101 consensus_queue.cc:237] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:50.646512 23025 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:50.646660 23351 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:50.646687 23162 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:50.646854 23162 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:50.647349 23158 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.647557 23351 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.648371 22834 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 5)
I20251028 21:12:50.648329 23025 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.648339 22846 catalog_manager.cc:5649] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:50.649401 23101 consensus_queue.cc:237] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.649710 23351 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:50.649714 23025 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:50.649863 23162 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:50.650002 23162 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:50.650333 23212 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.650473 23351 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.650588 23025 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.651330 22834 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:50.651361 22847 catalog_manager.cc:5649] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 reported cstate change: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:50.653777 23263 tablet_service.cc:1552] Processing DeleteTablet for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 with delete_type TABLET_DATA_TOMBSTONED (TS 9c8bcb21e08e4d9fb3e681731549601b not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:56334
W20251028 21:12:50.654093 22831 catalog_manager.cc:4952] TS 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4:42181): delete failed for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 because the tablet was not found. No further retry: Not found: Tablet not found: a1aa778fb70c461fa81bcc7cd2dc3db1
I20251028 21:12:50.692181 23101 consensus_queue.cc:237] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:50.692696 23351 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:50.692770 23025 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251028 21:12:50.692838 23064 consensus_peers.cc:597] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 -> Peer 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4:42181): Couldn't send request to peer 9c8bcb21e08e4d9fb3e681731549601b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6d1954a45164b32b866a241b9487e59. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:50.692878 23158 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:50.692943 23162 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:50.693539 23162 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.693715 23351 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.693686 23025 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.694525 22834 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e6d1954a45164b32b866a241b9487e59 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 6)
I20251028 21:12:50.694607 22846 catalog_manager.cc:5649] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:50.695353 23101 consensus_queue.cc:237] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.695652 23351 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:50.695648 23025 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:50.695822 23158 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:50.695888 23212 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:50.696350 23212 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.696518 23351 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.697170 22834 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet e6d1954a45164b32b866a241b9487e59 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:50.697278 22847 catalog_manager.cc:5649] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 reported cstate change: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:50.697345 23025 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.698148 23263 tablet_service.cc:1552] Processing DeleteTablet for tablet e6d1954a45164b32b866a241b9487e59 with delete_type TABLET_DATA_TOMBSTONED (TS 9c8bcb21e08e4d9fb3e681731549601b not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:56334
W20251028 21:12:50.698334 22831 catalog_manager.cc:4952] TS 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4:42181): delete failed for tablet e6d1954a45164b32b866a241b9487e59 because the tablet was not found. No further retry: Not found: Tablet not found: e6d1954a45164b32b866a241b9487e59
I20251028 21:12:50.704599 23216 consensus_queue.cc:579] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Leader has been unable to successfully communicate with peer 3a25731407814b3bb1a4125abce11bbd for more than 1 seconds (1.500s)
I20251028 21:12:50.705246 23025 consensus_queue.cc:237] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:50.705721 23101 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:50.705726 23351 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 21:12:50.705902 23399 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:50.706094 23401 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:50.706650 23399 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.706796 23351 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.706779 23101 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.707726 22847 catalog_manager.cc:5649] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd reported cstate change: config changed from index -1 to 2, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:50.707978 22833 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 9b3c763a0a8d441284b698fe0949090d with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251028 21:12:50.708521 23025 consensus_queue.cc:237] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } }
I20251028 21:12:50.708866 23351 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:50.708910 23101 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251028 21:12:50.709021 23216 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:50.709096 23399 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251028 21:12:50.709654 23399 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.709784 23101 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.709846 23351 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } }
I20251028 21:12:50.710400 22833 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 9b3c763a0a8d441284b698fe0949090d with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:50.710563 22847 catalog_manager.cc:5649] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e reported cstate change: config changed from index 2 to 3, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) evicted. New cstate: current_term: 1 leader_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } }
I20251028 21:12:50.711228 23263 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b3c763a0a8d441284b698fe0949090d with delete_type TABLET_DATA_TOMBSTONED (TS 9c8bcb21e08e4d9fb3e681731549601b not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:56334
W20251028 21:12:50.711409 22831 catalog_manager.cc:4952] TS 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4:42181): delete failed for tablet 9b3c763a0a8d441284b698fe0949090d because the tablet was not found. No further retry: Not found: Tablet not found: 9b3c763a0a8d441284b698fe0949090d
I20251028 21:12:51.085551 23025 raft_consensus.cc:1275] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 2 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251028 21:12:51.085829 23158 consensus_queue.cc:1048] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:51.087013 23351 raft_consensus.cc:1275] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 2 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251028 21:12:51.087204 23212 consensus_queue.cc:1048] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251028 21:12:51.528975 23299 heartbeater.cc:499] Master 127.15.205.62:33985 was elected leader, sending a full tablet report...
W20251028 21:12:51.542665 22896 auto_rebalancer.cc:249] could not retrieve auto-rebalancing replica moves: Not found: table cd63739fa16141fb9abb319114098251: could not find any suitable replica to move from server bb4ab48d916944119f57263513c12fb8 to server 9c8bcb21e08e4d9fb3e681731549601b
I20251028 21:12:52.545682 23025 consensus_queue.cc:237] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
W20251028 21:12:52.546275 22989 consensus_peers.cc:597] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e -> Peer 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4:42181): Couldn't send request to peer 9c8bcb21e08e4d9fb3e681731549601b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b3c763a0a8d441284b698fe0949090d. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:52.546253 23101 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:52.546275 23351 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:52.546502 23414 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:52.546607 23413 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:52.547297 23414 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.547421 23101 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.547425 23351 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.548560 22845 catalog_manager.cc:5649] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 reported cstate change: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.550959 23101 consensus_queue.cc:237] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:52.551474 23351 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:52.551474 23025 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20251028 21:12:52.551584 23064 consensus_peers.cc:597] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 -> Peer 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4:42181): Couldn't send request to peer 9c8bcb21e08e4d9fb3e681731549601b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6d1954a45164b32b866a241b9487e59. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:52.551680 23425 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:52.551764 23212 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:52.552376 23425 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.552474 23025 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.552585 23351 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.553313 22847 catalog_manager.cc:5649] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd reported cstate change: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.553704 23101 consensus_queue.cc:237] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:52.554126 23025 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251028 21:12:52.554153 23351 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20251028 21:12:52.554306 23064 consensus_peers.cc:597] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 -> Peer 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4:42181): Couldn't send request to peer 9c8bcb21e08e4d9fb3e681731549601b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a1aa778fb70c461fa81bcc7cd2dc3db1. This is attempt 1: this message will repeat every 5th retry.
I20251028 21:12:52.554351 23424 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:52.554456 23212 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251028 21:12:52.555023 23212 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.555128 23025 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.555158 23351 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.556073 22845 catalog_manager.cc:5649] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e reported cstate change: config changed from index 3 to 4, NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) added. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } } }
I20251028 21:12:52.964560 23431 ts_tablet_manager.cc:933] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: Initiating tablet copy from peer bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:52.964783 23431 tablet_copy_client.cc:323] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:45395
I20251028 21:12:52.968086 23111 tablet_copy_service.cc:140] P bb4ab48d916944119f57263513c12fb8: Received BeginTabletCopySession request for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 from peer 9c8bcb21e08e4d9fb3e681731549601b ({username='slave'} at 127.0.0.1:34776)
I20251028 21:12:52.968173 23111 tablet_copy_service.cc:161] P bb4ab48d916944119f57263513c12fb8: Beginning new tablet copy session on tablet a1aa778fb70c461fa81bcc7cd2dc3db1 from peer 9c8bcb21e08e4d9fb3e681731549601b at {username='slave'} at 127.0.0.1:34776: session id = 9c8bcb21e08e4d9fb3e681731549601b-a1aa778fb70c461fa81bcc7cd2dc3db1
I20251028 21:12:52.968894 23111 tablet_copy_source_session.cc:215] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:52.969251 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1aa778fb70c461fa81bcc7cd2dc3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:52.970579 23431 tablet_copy_client.cc:806] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:52.970690 23431 tablet_copy_client.cc:670] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:52.971117 23431 tablet_copy_client.cc:538] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:52.971791 23431 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap starting.
I20251028 21:12:52.973191 23431 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:52.973369 23431 tablet_bootstrap.cc:492] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap complete.
I20251028 21:12:52.973466 23431 ts_tablet_manager.cc:1403] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 21:12:52.973618 23431 raft_consensus.cc:359] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:52.973706 23431 raft_consensus.cc:740] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9c8bcb21e08e4d9fb3e681731549601b, State: Initialized, Role: LEARNER
I20251028 21:12:52.973737 23431 consensus_queue.cc:260] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:52.973902 23431 ts_tablet_manager.cc:1434] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:52.974113 23111 tablet_copy_service.cc:342] P bb4ab48d916944119f57263513c12fb8: Request end of tablet copy session 9c8bcb21e08e4d9fb3e681731549601b-a1aa778fb70c461fa81bcc7cd2dc3db1 received from {username='slave'} at 127.0.0.1:34776
I20251028 21:12:52.974164 23111 tablet_copy_service.cc:434] P bb4ab48d916944119f57263513c12fb8: ending tablet copy session 9c8bcb21e08e4d9fb3e681731549601b-a1aa778fb70c461fa81bcc7cd2dc3db1 on tablet a1aa778fb70c461fa81bcc7cd2dc3db1 with peer 9c8bcb21e08e4d9fb3e681731549601b
I20251028 21:12:53.011129 23431 ts_tablet_manager.cc:933] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: Initiating tablet copy from peer bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395)
I20251028 21:12:53.011330 23431 tablet_copy_client.cc:323] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.3:45395
I20251028 21:12:53.011533 23111 tablet_copy_service.cc:140] P bb4ab48d916944119f57263513c12fb8: Received BeginTabletCopySession request for tablet e6d1954a45164b32b866a241b9487e59 from peer 9c8bcb21e08e4d9fb3e681731549601b ({username='slave'} at 127.0.0.1:34776)
I20251028 21:12:53.011588 23111 tablet_copy_service.cc:161] P bb4ab48d916944119f57263513c12fb8: Beginning new tablet copy session on tablet e6d1954a45164b32b866a241b9487e59 from peer 9c8bcb21e08e4d9fb3e681731549601b at {username='slave'} at 127.0.0.1:34776: session id = 9c8bcb21e08e4d9fb3e681731549601b-e6d1954a45164b32b866a241b9487e59
I20251028 21:12:53.012104 23111 tablet_copy_source_session.cc:215] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:53.012364 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6d1954a45164b32b866a241b9487e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:53.013516 23431 tablet_copy_client.cc:806] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:53.013669 23431 tablet_copy_client.cc:670] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:53.014130 23431 tablet_copy_client.cc:538] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:53.014766 23431 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap starting.
I20251028 21:12:53.016264 23431 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:53.016430 23431 tablet_bootstrap.cc:492] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap complete.
I20251028 21:12:53.016516 23431 ts_tablet_manager.cc:1403] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20251028 21:12:53.016685 23431 raft_consensus.cc:359] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:53.016768 23431 raft_consensus.cc:740] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9c8bcb21e08e4d9fb3e681731549601b, State: Initialized, Role: LEARNER
I20251028 21:12:53.016811 23431 consensus_queue.cc:260] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:53.016901 23431 ts_tablet_manager.cc:1434] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251028 21:12:53.017077 23111 tablet_copy_service.cc:342] P bb4ab48d916944119f57263513c12fb8: Request end of tablet copy session 9c8bcb21e08e4d9fb3e681731549601b-e6d1954a45164b32b866a241b9487e59 received from {username='slave'} at 127.0.0.1:34776
I20251028 21:12:53.017120 23111 tablet_copy_service.cc:434] P bb4ab48d916944119f57263513c12fb8: ending tablet copy session 9c8bcb21e08e4d9fb3e681731549601b-e6d1954a45164b32b866a241b9487e59 on tablet e6d1954a45164b32b866a241b9487e59 with peer 9c8bcb21e08e4d9fb3e681731549601b
I20251028 21:12:53.071489 23431 ts_tablet_manager.cc:933] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: Initiating tablet copy from peer 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217)
I20251028 21:12:53.071648 23431 tablet_copy_client.cc:323] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.205.2:44217
I20251028 21:12:53.074565 23036 tablet_copy_service.cc:140] P 7b24afaeee5d4b2388b84d94ea28a92e: Received BeginTabletCopySession request for tablet 9b3c763a0a8d441284b698fe0949090d from peer 9c8bcb21e08e4d9fb3e681731549601b ({username='slave'} at 127.0.0.1:54524)
I20251028 21:12:53.074630 23036 tablet_copy_service.cc:161] P 7b24afaeee5d4b2388b84d94ea28a92e: Beginning new tablet copy session on tablet 9b3c763a0a8d441284b698fe0949090d from peer 9c8bcb21e08e4d9fb3e681731549601b at {username='slave'} at 127.0.0.1:54524: session id = 9c8bcb21e08e4d9fb3e681731549601b-9b3c763a0a8d441284b698fe0949090d
I20251028 21:12:53.075212 23036 tablet_copy_source_session.cc:215] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e: Tablet Copy: opened 0 blocks and 1 log segments
I20251028 21:12:53.075520 23431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3c763a0a8d441284b698fe0949090d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 21:12:53.076413 23431 tablet_copy_client.cc:806] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Starting download of 0 data blocks...
I20251028 21:12:53.076514 23431 tablet_copy_client.cc:670] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Starting download of 1 WAL segments...
I20251028 21:12:53.076982 23431 tablet_copy_client.cc:538] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251028 21:12:53.077492 23431 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap starting.
I20251028 21:12:53.078698 23431 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 21:12:53.078867 23431 tablet_bootstrap.cc:492] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: Bootstrap complete.
I20251028 21:12:53.078971 23431 ts_tablet_manager.cc:1403] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251028 21:12:53.079137 23431 raft_consensus.cc:359] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:53.079205 23431 raft_consensus.cc:740] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9c8bcb21e08e4d9fb3e681731549601b, State: Initialized, Role: LEARNER
I20251028 21:12:53.079234 23431 consensus_queue.cc:260] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: NON_VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: true } }
I20251028 21:12:53.079358 23431 ts_tablet_manager.cc:1434] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251028 21:12:53.079574 23036 tablet_copy_service.cc:342] P 7b24afaeee5d4b2388b84d94ea28a92e: Request end of tablet copy session 9c8bcb21e08e4d9fb3e681731549601b-9b3c763a0a8d441284b698fe0949090d received from {username='slave'} at 127.0.0.1:54524
I20251028 21:12:53.079622 23036 tablet_copy_service.cc:434] P 7b24afaeee5d4b2388b84d94ea28a92e: ending tablet copy session 9c8bcb21e08e4d9fb3e681731549601b-9b3c763a0a8d441284b698fe0949090d on tablet 9b3c763a0a8d441284b698fe0949090d with peer 9c8bcb21e08e4d9fb3e681731549601b
I20251028 21:12:53.462867 23273 raft_consensus.cc:1217] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251028 21:12:53.562047 23273 raft_consensus.cc:1217] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251028 21:12:53.563442 23273 raft_consensus.cc:1217] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251028 21:12:53.888154 23441 raft_consensus.cc:1064] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e: attempting to promote NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b to VOTER
I20251028 21:12:53.888408 23441 consensus_queue.cc:237] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } }
I20251028 21:12:53.889027 23100 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:53.889003 23273 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:53.889017 23351 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:53.889292 23413 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:53.889382 23444 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:53.889542 23441 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:53.890269 23413 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:53.890367 23101 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:53.891220 23351 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:53.891420 22847 catalog_manager.cc:5649] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e reported cstate change: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251028 21:12:53.891579 23273 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } attrs { replace: true } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:53.893649 23025 consensus_queue.cc:237] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } }
I20251028 21:12:53.893949 23273 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251028 21:12:53.894017 23351 raft_consensus.cc:1275] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer 7b24afaeee5d4b2388b84d94ea28a92e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251028 21:12:53.894174 23444 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251028 21:12:53.894304 23445 consensus_queue.cc:1048] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251028 21:12:53.894630 23413 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bb4ab48d916944119f57263513c12fb8 (127.15.205.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:53.894716 23273 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bb4ab48d916944119f57263513c12fb8 (127.15.205.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:53.895401 22833 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 9b3c763a0a8d441284b698fe0949090d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:53.895555 22846 catalog_manager.cc:5649] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b reported cstate change: config changed from index 5 to 6, VOTER bb4ab48d916944119f57263513c12fb8 (127.15.205.3) evicted. New cstate: current_term: 1 leader_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:53.895752 23351 raft_consensus.cc:2955] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER bb4ab48d916944119f57263513c12fb8 (127.15.205.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:53.896128 23090 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b3c763a0a8d441284b698fe0949090d with delete_type TABLET_DATA_TOMBSTONED (TS bb4ab48d916944119f57263513c12fb8 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34750
I20251028 21:12:53.896315 23454 tablet_replica.cc:333] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: stopping tablet replica
I20251028 21:12:53.896484 23454 raft_consensus.cc:2243] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:53.896600 23454 raft_consensus.cc:2272] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:53.896839 23454 ts_tablet_manager.cc:1916] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:12:53.897930 23454 ts_tablet_manager.cc:1929] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251028 21:12:53.897980 23454 log.cc:1199] T 9b3c763a0a8d441284b698fe0949090d P bb4ab48d916944119f57263513c12fb8: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-2-root/wals/9b3c763a0a8d441284b698fe0949090d
I20251028 21:12:53.898286 22834 catalog_manager.cc:4985] TS bb4ab48d916944119f57263513c12fb8 (127.15.205.3:45395): tablet 9b3c763a0a8d441284b698fe0949090d (table test-workload [id=cd63739fa16141fb9abb319114098251]) successfully deleted
I20251028 21:12:54.086900 23442 raft_consensus.cc:1064] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8: attempting to promote NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b to VOTER
I20251028 21:12:54.087138 23442 consensus_queue.cc:237] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } }
I20251028 21:12:54.087697 23025 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:54.087724 23351 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:54.087724 23273 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:54.087899 23458 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:54.088043 23457 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:54.088279 23455 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:54.089107 23455 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.089247 23351 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.089263 23273 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.090270 23025 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.090306 22844 catalog_manager.cc:5649] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 reported cstate change: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:12:54.091437 23101 consensus_queue.cc:237] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } }
I20251028 21:12:54.091750 23025 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251028 21:12:54.091845 23273 raft_consensus.cc:1275] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251028 21:12:54.091922 23455 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251028 21:12:54.092022 23442 consensus_queue.cc:1048] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251028 21:12:54.092365 23458 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3a25731407814b3bb1a4125abce11bbd (127.15.205.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.092448 23025 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3a25731407814b3bb1a4125abce11bbd (127.15.205.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.092640 23273 raft_consensus.cc:2955] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3a25731407814b3bb1a4125abce11bbd (127.15.205.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.093374 22834 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet e6d1954a45164b32b866a241b9487e59 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:54.093379 22846 catalog_manager.cc:5649] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 reported cstate change: config changed from index 5 to 6, VOTER 3a25731407814b3bb1a4125abce11bbd (127.15.205.1) evicted. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251028 21:12:54.094103 23342 tablet_service.cc:1552] Processing DeleteTablet for tablet e6d1954a45164b32b866a241b9487e59 with delete_type TABLET_DATA_TOMBSTONED (TS 3a25731407814b3bb1a4125abce11bbd not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39452
I20251028 21:12:54.094480 23462 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:54.094544 23462 raft_consensus.cc:2243] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:54.094611 23462 raft_consensus.cc:2272] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:54.094816 23462 ts_tablet_manager.cc:1916] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:12:54.096015 23462 ts_tablet_manager.cc:1929] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251028 21:12:54.096093 23462 log.cc:1199] T e6d1954a45164b32b866a241b9487e59 P 3a25731407814b3bb1a4125abce11bbd: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-0-root/wals/e6d1954a45164b32b866a241b9487e59
I20251028 21:12:54.096442 22834 catalog_manager.cc:4985] TS 3a25731407814b3bb1a4125abce11bbd (127.15.205.1:41405): tablet e6d1954a45164b32b866a241b9487e59 (table test-workload [id=cd63739fa16141fb9abb319114098251]) successfully deleted
I20251028 21:12:54.110672 23457 raft_consensus.cc:1064] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8: attempting to promote NON_VOTER 9c8bcb21e08e4d9fb3e681731549601b to VOTER
I20251028 21:12:54.110839 23457 consensus_queue.cc:237] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } }
I20251028 21:12:54.111250 23273 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 LEARNER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:54.111300 23351 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:54.111362 23025 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251028 21:12:54.111459 23455 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:54.111541 23458 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:54.111608 23442 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 21:12:54.112394 23458 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.112462 23025 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.113272 23351 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.113274 23273 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.113370 22844 catalog_manager.cc:5649] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e reported cstate change: config changed from index 4 to 5, 9c8bcb21e08e4d9fb3e681731549601b (127.15.205.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "7b24afaeee5d4b2388b84d94ea28a92e" member_type: VOTER last_known_addr { host: "127.15.205.2" port: 44217 } attrs { replace: true } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.114418 23100 consensus_queue.cc:237] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } }
I20251028 21:12:54.114818 23273 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251028 21:12:54.114825 23351 raft_consensus.cc:1275] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Refusing update from remote peer bb4ab48d916944119f57263513c12fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251028 21:12:54.114981 23442 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251028 21:12:54.115051 23457 consensus_queue.cc:1048] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251028 21:12:54.115542 23455 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.115592 23351 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.115610 23273 raft_consensus.cc:2955] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.116379 22834 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251028 21:12:54.116458 22844 catalog_manager.cc:5649] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd reported cstate change: config changed from index 5 to 6, VOTER 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2) evicted. New cstate: current_term: 1 leader_uuid: "bb4ab48d916944119f57263513c12fb8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3a25731407814b3bb1a4125abce11bbd" member_type: VOTER last_known_addr { host: "127.15.205.1" port: 41405 } } peers { permanent_uuid: "bb4ab48d916944119f57263513c12fb8" member_type: VOTER last_known_addr { host: "127.15.205.3" port: 45395 } } peers { permanent_uuid: "9c8bcb21e08e4d9fb3e681731549601b" member_type: VOTER last_known_addr { host: "127.15.205.4" port: 42181 } attrs { promote: false } } }
I20251028 21:12:54.117063 23016 tablet_service.cc:1552] Processing DeleteTablet for tablet a1aa778fb70c461fa81bcc7cd2dc3db1 with delete_type TABLET_DATA_TOMBSTONED (TS 7b24afaeee5d4b2388b84d94ea28a92e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54498
I20251028 21:12:54.117223 23467 tablet_replica.cc:333] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: stopping tablet replica
I20251028 21:12:54.117271 23467 raft_consensus.cc:2243] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:54.117319 23467 raft_consensus.cc:2272] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:54.117511 23467 ts_tablet_manager.cc:1916] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251028 21:12:54.118484 23467 ts_tablet_manager.cc:1929] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251028 21:12:54.118530 23467 log.cc:1199] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 7b24afaeee5d4b2388b84d94ea28a92e: Deleting WAL directory at /tmp/dist-test-task4ah29Q/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1761685792944951-16180-0/minicluster-data/ts-1-root/wals/a1aa778fb70c461fa81bcc7cd2dc3db1
I20251028 21:12:54.118832 22833 catalog_manager.cc:4985] TS 7b24afaeee5d4b2388b84d94ea28a92e (127.15.205.2:44217): tablet a1aa778fb70c461fa81bcc7cd2dc3db1 (table test-workload [id=cd63739fa16141fb9abb319114098251]) successfully deleted
W20251028 21:12:55.116828 22896 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver bb4ab48d916944119f57263513c12fb8 not available for placement
I20251028 21:12:56.085422 16180 tablet_server.cc:178] TabletServer@127.15.205.1:41405 shutting down...
I20251028 21:12:56.087751 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:56.087845 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:56.087904 16180 raft_consensus.cc:2243] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:56.087944 16180 raft_consensus.cc:2272] T 9f9a28ac41a04f3990922179532c2ca9 P 3a25731407814b3bb1a4125abce11bbd [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.088085 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:56.088110 16180 raft_consensus.cc:2243] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:56.088155 16180 raft_consensus.cc:2272] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.088238 16180 tablet_replica.cc:333] stopping tablet replica
I20251028 21:12:56.088263 16180 raft_consensus.cc:2243] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:56.088300 16180 raft_consensus.cc:2272] T 9b3c763a0a8d441284b698fe0949090d P 3a25731407814b3bb1a4125abce11bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.099835 16180 tablet_server.cc:195] TabletServer@127.15.205.1:41405 shutdown complete.
I20251028 21:12:56.101054 16180 tablet_server.cc:178] TabletServer@127.15.205.2:0 shutting down...
I20251028 21:12:56.103929 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:56.104038 16180 tablet_replica.cc:333] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e: stopping tablet replica
I20251028 21:12:56.104091 16180 raft_consensus.cc:2243] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:56.104162 16180 raft_consensus.cc:2272] T e6d1954a45164b32b866a241b9487e59 P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.104297 16180 tablet_replica.cc:333] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e: stopping tablet replica
I20251028 21:12:56.104331 16180 raft_consensus.cc:2243] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:56.104404 16180 raft_consensus.cc:2272] T 9f9a28ac41a04f3990922179532c2ca9 P 7b24afaeee5d4b2388b84d94ea28a92e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.104512 16180 tablet_replica.cc:333] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e: stopping tablet replica
I20251028 21:12:56.104543 16180 raft_consensus.cc:2243] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:56.104598 16180 raft_consensus.cc:2272] T 9b3c763a0a8d441284b698fe0949090d P 7b24afaeee5d4b2388b84d94ea28a92e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.116214 16180 tablet_server.cc:195] TabletServer@127.15.205.2:0 shutdown complete.
W20251028 21:12:56.117043 22896 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 7b24afaeee5d4b2388b84d94ea28a92e not available for placement
I20251028 21:12:56.117317 16180 tablet_server.cc:178] TabletServer@127.15.205.3:0 shutting down...
I20251028 21:12:56.120699 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:56.120826 16180 tablet_replica.cc:333] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8: stopping tablet replica
I20251028 21:12:56.120893 16180 raft_consensus.cc:2243] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:56.120952 16180 raft_consensus.cc:2272] T a1aa778fb70c461fa81bcc7cd2dc3db1 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.121057 16180 tablet_replica.cc:333] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8: stopping tablet replica
I20251028 21:12:56.121081 16180 raft_consensus.cc:2243] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 2 LEADER]: Raft consensus shutting down.
I20251028 21:12:56.121114 16180 raft_consensus.cc:2272] T 9f9a28ac41a04f3990922179532c2ca9 P bb4ab48d916944119f57263513c12fb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.121191 16180 tablet_replica.cc:333] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8: stopping tablet replica
I20251028 21:12:56.121214 16180 raft_consensus.cc:2243] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:56.121245 16180 raft_consensus.cc:2272] T e6d1954a45164b32b866a241b9487e59 P bb4ab48d916944119f57263513c12fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.132694 16180 tablet_server.cc:195] TabletServer@127.15.205.3:0 shutdown complete.
I20251028 21:12:56.133638 16180 tablet_server.cc:178] TabletServer@127.15.205.4:0 shutting down...
I20251028 21:12:56.135799 16180 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251028 21:12:56.135933 16180 tablet_replica.cc:333] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b: stopping tablet replica
I20251028 21:12:56.136013 16180 raft_consensus.cc:2243] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:56.136066 16180 raft_consensus.cc:2272] T a1aa778fb70c461fa81bcc7cd2dc3db1 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.136202 16180 tablet_replica.cc:333] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b: stopping tablet replica
I20251028 21:12:56.136236 16180 raft_consensus.cc:2243] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:56.136278 16180 raft_consensus.cc:2272] T 9b3c763a0a8d441284b698fe0949090d P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.136381 16180 tablet_replica.cc:333] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b: stopping tablet replica
I20251028 21:12:56.136411 16180 raft_consensus.cc:2243] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 21:12:56.136453 16180 raft_consensus.cc:2272] T e6d1954a45164b32b866a241b9487e59 P 9c8bcb21e08e4d9fb3e681731549601b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.147611 16180 tablet_server.cc:195] TabletServer@127.15.205.4:0 shutdown complete.
I20251028 21:12:56.148401 16180 master.cc:561] Master@127.15.205.62:33985 shutting down...
I20251028 21:12:56.150972 16180 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 1 LEADER]: Raft consensus shutting down.
I20251028 21:12:56.151024 16180 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 21:12:56.151046 16180 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6b8026167662446e846ba07fa0042298: stopping tablet replica
I20251028 21:12:56.163085 16180 master.cc:583] Master@127.15.205.62:33985 shutdown complete.
[ OK ] AutoRebalancerTest.TestDeletedTables (8658 ms)
[----------] 14 tests from AutoRebalancerTest (183219 ms total)
[----------] Global test environment tear-down
[==========] 14 tests from 1 test suite ran. (183219 ms total)
[ PASSED ] 13 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer
1 FAILED TEST
I20251028 21:12:56.168881 16180 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 7 messages since previous log ~5 seconds ago
I20251028 21:12:56.168913 16180 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/ts_tablet_manager.cc:542: suppressed but not reported on 7 messages since previous log ~5 seconds ago
I20251028 21:12:56.168944 16180 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/catalog_manager.cc:4712: suppressed but not reported on 7 messages since previous log ~5 seconds ago
I20251028 21:12:56.168980 16180 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/catalog_manager.cc:5327: suppressed but not reported on 9 messages since previous log ~30 seconds ago
I20251028 21:12:56.169011 16180 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 12 messages since previous log ~171 seconds ago
I20251028 21:12:56.169034 16180 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 37 messages since previous log ~12 seconds ago
I20251028 21:12:56.169106 16180 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 18 messages since previous log ~8 seconds ago