[==========] Running 14 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 14 tests from AutoRebalancerTest
[ RUN ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251209 03:02:36.541640 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:33821,127.2.170.253:42009,127.2.170.252:37313
I20251209 03:02:36.541956 2731 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251209 03:02:36.542061 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:36.545843 2739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:36.545832 2736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.545864 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:36.545984 2737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.546211 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:36.546255 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:36.546272 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249356546272 us; error 0 us; skew 500 ppm
I20251209 03:02:36.547250 2731 webserver.cc:492] Webserver started at http://127.2.170.254:37929/ using document root <none> and password file <none>
I20251209 03:02:36.547369 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:36.547408 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:36.547487 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:36.548420 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "00f1195cb8c548618455d7affd4dedd9"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.549453 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20251209 03:02:36.549949 2744 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.550094 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.550169 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "00f1195cb8c548618455d7affd4dedd9"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.550220 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:36.567772 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:36.567991 2731 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251209 03:02:36.568077 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:36.571672 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:33821
I20251209 03:02:36.571696 2796 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:33821 every 8 connection(s)
I20251209 03:02:36.572052 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:02:36.572172 2797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251209 03:02:36.573074 2799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:36.573120 2800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.573311 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:36.573421 2802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.573587 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:36.573627 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:36.573638 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249356573638 us; error 0 us; skew 500 ppm
I20251209 03:02:36.574138 2731 webserver.cc:492] Webserver started at http://127.2.170.253:42835/ using document root <none> and password file <none>
I20251209 03:02:36.574196 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:36.574234 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:36.574277 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:36.574470 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root/instance:
uuid: "dd3f55b6384b451f9b28474bfd2c1cb3"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.574611 2797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:36.575311 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:36.575783 2809 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.575882 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.575939 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root
uuid: "dd3f55b6384b451f9b28474bfd2c1cb3"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.575980 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:36.579530 2797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } has no permanent_uuid. Determining permanent_uuid...
W20251209 03:02:36.579825 2748 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.170.253:42009: connect: Connection refused (error 111)
W20251209 03:02:36.580153 2797 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.170.253:42009: Network error: Client connection negotiation failed: client connection to 127.2.170.253:42009: connect: Connection refused (error 111)
I20251209 03:02:36.583513 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:36.583691 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:36.586706 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.253:42009
I20251209 03:02:36.586756 2862 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.253:42009 every 8 connection(s)
I20251209 03:02:36.586964 2863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:36.586973 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:02:36.587582 2863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } has no permanent_uuid. Determining permanent_uuid...
W20251209 03:02:36.588056 2865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:36.588302 2866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:36.588555 2868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.588644 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:36.588766 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:36.588793 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:36.588809 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249356588809 us; error 0 us; skew 500 ppm
I20251209 03:02:36.589294 2731 webserver.cc:492] Webserver started at http://127.2.170.252:38437/ using document root <none> and password file <none>
I20251209 03:02:36.589365 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:36.589403 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:36.589442 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:36.589709 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root/instance:
uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.590607 2863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:36.590687 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:36.591189 2875 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.591295 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.591336 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root
uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.591367 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:36.593461 2863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } has no permanent_uuid. Determining permanent_uuid...
W20251209 03:02:36.593988 2863 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.170.252:37313: Network error: Client connection negotiation failed: client connection to 127.2.170.252:37313: connect: Connection refused (error 111)
I20251209 03:02:36.600035 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:36.600209 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:36.603168 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.252:37313
I20251209 03:02:36.603210 2928 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.252:37313 every 8 connection(s)
I20251209 03:02:36.603328 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:02:36.603410 2929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:36.603916 2929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:36.606113 2929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:36.608130 2929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:36.611160 2929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77: Bootstrap starting.
I20251209 03:02:36.611860 2929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:36.612089 2929 log.cc:826] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77: Log is configured to *not* fsync() on all Append() calls
I20251209 03:02:36.612582 2929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77: No bootstrap required, opened a new log
I20251209 03:02:36.613586 2929 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.613698 2929 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:36.613734 2929 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8c9e8f2a5c49038e3b2e46994f5f77, State: Initialized, Role: FOLLOWER
I20251209 03:02:36.613816 2929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.614007 2934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.614051 2929 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:02:36.614080 2934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: This master's current role is: FOLLOWER
W20251209 03:02:36.615399 2945 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251209 03:02:36.615439 2945 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251209 03:02:36.629292 2797 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } attempt: 1
I20251209 03:02:36.631775 2797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:36.634941 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9: Bootstrap starting.
I20251209 03:02:36.635320 2797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:36.635807 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9: No bootstrap required, opened a new log
I20251209 03:02:36.635951 2797 raft_consensus.cc:359] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.635999 2797 raft_consensus.cc:385] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:36.636013 2797 raft_consensus.cc:740] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f1195cb8c548618455d7affd4dedd9, State: Initialized, Role: FOLLOWER
I20251209 03:02:36.636044 2797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.636197 2797 sys_catalog.cc:565] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:02:36.636204 2948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.636322 2948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:36.637342 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251209 03:02:36.637465 2959 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251209 03:02:36.637503 2959 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251209 03:02:36.646116 2863 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } attempt: 1
I20251209 03:02:36.648909 2863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3: Bootstrap starting.
I20251209 03:02:36.649235 2863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:36.649816 2863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3: No bootstrap required, opened a new log
I20251209 03:02:36.649976 2863 raft_consensus.cc:359] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.650027 2863 raft_consensus.cc:385] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:36.650043 2863 raft_consensus.cc:740] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd3f55b6384b451f9b28474bfd2c1cb3, State: Initialized, Role: FOLLOWER
I20251209 03:02:36.650084 2863 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.650256 2863 sys_catalog.cc:565] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:02:36.650265 2961 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.650389 2961 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:36.651505 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20251209 03:02:36.651793 2972 catalog_manager.cc:1568] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251209 03:02:36.651844 2972 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251209 03:02:36.651798 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:36.653149 2974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:36.653201 2973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.653311 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:36.653784 2976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.653949 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:36.653990 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:36.654002 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249356654002 us; error 0 us; skew 500 ppm
I20251209 03:02:36.654496 2731 webserver.cc:492] Webserver started at http://127.2.170.193:37899/ using document root <none> and password file <none>
I20251209 03:02:36.654559 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:36.654597 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:36.654640 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:36.654863 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "6d7b2d74b8384bd3bbd73c9792491cb9"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.655678 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:36.656342 2981 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.656452 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.656489 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "6d7b2d74b8384bd3bbd73c9792491cb9"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.656530 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:36.666548 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:36.666781 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:36.666955 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:36.667249 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:36.667294 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.667327 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:36.667353 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.671550 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:35077
I20251209 03:02:36.671602 3044 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:35077 every 8 connection(s)
I20251209 03:02:36.671990 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:36.673377 3053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:36.673799 3055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.674015 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:36.675283 3052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.675668 3045 heartbeater.cc:344] Connected to a master server at 127.2.170.252:37313
I20251209 03:02:36.675786 3045 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.675972 3046 heartbeater.cc:344] Connected to a master server at 127.2.170.254:33821
I20251209 03:02:36.676040 3046 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.675990 3045 heartbeater.cc:507] Master 127.2.170.252:37313 requested a full tablet report, sending...
I20251209 03:02:36.676187 3046 heartbeater.cc:507] Master 127.2.170.254:33821 requested a full tablet report, sending...
I20251209 03:02:36.676601 2893 ts_manager.cc:194] Registered new tserver with Master: 6d7b2d74b8384bd3bbd73c9792491cb9 (127.2.170.193:35077)
I20251209 03:02:36.676606 2761 ts_manager.cc:194] Registered new tserver with Master: 6d7b2d74b8384bd3bbd73c9792491cb9 (127.2.170.193:35077)
I20251209 03:02:36.676906 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:36.676949 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:36.676966 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249356676966 us; error 0 us; skew 500 ppm
I20251209 03:02:36.677778 2731 webserver.cc:492] Webserver started at http://127.2.170.194:36775/ using document root <none> and password file <none>
I20251209 03:02:36.677860 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:36.677888 3047 heartbeater.cc:344] Connected to a master server at 127.2.170.253:42009
I20251209 03:02:36.677953 3047 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.677904 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:36.678025 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:36.678058 3047 heartbeater.cc:507] Master 127.2.170.253:42009 requested a full tablet report, sending...
I20251209 03:02:36.678260 2827 ts_manager.cc:194] Registered new tserver with Master: 6d7b2d74b8384bd3bbd73c9792491cb9 (127.2.170.193:35077)
I20251209 03:02:36.678284 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "ac3c9240e20643dd8648c5cedf32ef02"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.679234 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20251209 03:02:36.679687 3060 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.679808 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.679857 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "ac3c9240e20643dd8648c5cedf32ef02"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.679898 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:36.686686 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:36.686920 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:36.687067 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:36.687235 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:36.687268 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.687299 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:36.687322 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.691015 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:43941
I20251209 03:02:36.691076 3123 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:43941 every 8 connection(s)
I20251209 03:02:36.691444 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:02:36.694896 3124 heartbeater.cc:344] Connected to a master server at 127.2.170.252:37313
I20251209 03:02:36.694955 3124 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.695057 3124 heartbeater.cc:507] Master 127.2.170.252:37313 requested a full tablet report, sending...
I20251209 03:02:36.695309 3125 heartbeater.cc:344] Connected to a master server at 127.2.170.254:33821
I20251209 03:02:36.695353 3125 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.695387 2893 ts_manager.cc:194] Registered new tserver with Master: ac3c9240e20643dd8648c5cedf32ef02 (127.2.170.194:43941)
I20251209 03:02:36.695437 3125 heartbeater.cc:507] Master 127.2.170.254:33821 requested a full tablet report, sending...
W20251209 03:02:36.695539 3130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.695777 2761 ts_manager.cc:194] Registered new tserver with Master: ac3c9240e20643dd8648c5cedf32ef02 (127.2.170.194:43941)
I20251209 03:02:36.696009 3126 heartbeater.cc:344] Connected to a master server at 127.2.170.253:42009
I20251209 03:02:36.696062 3126 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.696156 3126 heartbeater.cc:507] Master 127.2.170.253:42009 requested a full tablet report, sending...
I20251209 03:02:36.696359 2827 ts_manager.cc:194] Registered new tserver with Master: ac3c9240e20643dd8648c5cedf32ef02 (127.2.170.194:43941)
W20251209 03:02:36.696547 3132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:36.696817 3134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:36.697240 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:36.697365 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:36.697398 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:36.697415 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249356697415 us; error 0 us; skew 500 ppm
I20251209 03:02:36.697945 2731 webserver.cc:492] Webserver started at http://127.2.170.195:46647/ using document root <none> and password file <none>
I20251209 03:02:36.698024 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:36.698065 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:36.698107 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:36.698347 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "a270733bce914c52859a7a1cc8ae5354"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.699314 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:36.699765 3139 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.699901 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.699945 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "a270733bce914c52859a7a1cc8ae5354"
format_stamp: "Formatted at 2025-12-09 03:02:36 on dist-test-slave-4cb0"
I20251209 03:02:36.699975 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:36.714064 2934 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:36.714161 2934 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.714367 2934 leader_election.cc:290] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00f1195cb8c548618455d7affd4dedd9 (127.2.170.254:33821), dd3f55b6384b451f9b28474bfd2c1cb3 (127.2.170.253:42009)
I20251209 03:02:36.714627 2771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f1195cb8c548618455d7affd4dedd9" is_pre_election: true
I20251209 03:02:36.714792 2771 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f8c9e8f2a5c49038e3b2e46994f5f77 in term 0.
I20251209 03:02:36.714625 2837 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" is_pre_election: true
I20251209 03:02:36.714929 2837 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f8c9e8f2a5c49038e3b2e46994f5f77 in term 0.
I20251209 03:02:36.714924 2878 leader_election.cc:304] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f1195cb8c548618455d7affd4dedd9, 7f8c9e8f2a5c49038e3b2e46994f5f77; no voters:
I20251209 03:02:36.715095 2934 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:36.715122 2934 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:36.715140 2934 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:36.715809 2934 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.715919 2934 leader_election.cc:290] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [CANDIDATE]: Term 1 election: Requested vote from peers 00f1195cb8c548618455d7affd4dedd9 (127.2.170.254:33821), dd3f55b6384b451f9b28474bfd2c1cb3 (127.2.170.253:42009)
I20251209 03:02:36.716058 2837 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3"
I20251209 03:02:36.716063 2771 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00f1195cb8c548618455d7affd4dedd9"
I20251209 03:02:36.716187 2771 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:36.716130 2837 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:36.716773 2771 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f8c9e8f2a5c49038e3b2e46994f5f77 in term 1.
I20251209 03:02:36.716890 2878 leader_election.cc:304] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00f1195cb8c548618455d7affd4dedd9, 7f8c9e8f2a5c49038e3b2e46994f5f77; no voters:
I20251209 03:02:36.716938 2934 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:36.716954 2837 raft_consensus.cc:2468] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f8c9e8f2a5c49038e3b2e46994f5f77 in term 1.
I20251209 03:02:36.717041 2934 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 1 LEADER]: Becoming Leader. State: Replica: 7f8c9e8f2a5c49038e3b2e46994f5f77, State: Running, Role: LEADER
I20251209 03:02:36.717105 2934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } }
I20251209 03:02:36.717469 3144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f8c9e8f2a5c49038e3b2e46994f5f77. Latest consensus state: current_term: 1 leader_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.717541 3144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:36.717706 3146 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:02:36.717864 3146 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:02:36.719039 2837 raft_consensus.cc:1275] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 1 FOLLOWER]: Refusing update from remote peer 7f8c9e8f2a5c49038e3b2e46994f5f77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:36.719040 2771 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 1 FOLLOWER]: Refusing update from remote peer 7f8c9e8f2a5c49038e3b2e46994f5f77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:36.719241 3144 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:36.719544 3144 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:36.719712 2948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f8c9e8f2a5c49038e3b2e46994f5f77. Latest consensus state: current_term: 1 leader_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.719784 2948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:36.720029 2961 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f8c9e8f2a5c49038e3b2e46994f5f77. Latest consensus state: current_term: 1 leader_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.720108 2961 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:36.720718 3144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.720778 3144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:36.720876 2934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.720948 2934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:36.721153 2961 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.721204 2961 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:36.721238 2948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00f1195cb8c548618455d7affd4dedd9" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 33821 } } peers { permanent_uuid: "dd3f55b6384b451f9b28474bfd2c1cb3" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42009 } } peers { permanent_uuid: "7f8c9e8f2a5c49038e3b2e46994f5f77" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 37313 } } }
I20251209 03:02:36.721268 2948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:36.721338 3148 mvcc.cc:204] Tried to move back new op lower bound from 7230461365119553536 to 7230461365113868288. Current Snapshot: MvccSnapshot[applied={T|T < 7230461365119553536}]
I20251209 03:02:36.721983 3146 catalog_manager.cc:1357] Generated new cluster ID: 6df2b828b8e94224978fc0f488591165
I20251209 03:02:36.722035 3146 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:02:36.735775 3146 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:02:36.736430 3146 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:02:36.745086 3146 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77: Generated new TSK 0
I20251209 03:02:36.745273 3146 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:02:36.750085 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:36.750308 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:36.750452 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:36.750658 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:36.750692 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.750715 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:36.750738 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:36.754851 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:43435
I20251209 03:02:36.754971 3214 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:43435 every 8 connection(s)
I20251209 03:02:36.758657 3216 heartbeater.cc:344] Connected to a master server at 127.2.170.254:33821
I20251209 03:02:36.758766 3216 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.758904 3216 heartbeater.cc:507] Master 127.2.170.254:33821 requested a full tablet report, sending...
I20251209 03:02:36.759171 3215 heartbeater.cc:344] Connected to a master server at 127.2.170.252:37313
I20251209 03:02:36.759182 2761 ts_manager.cc:194] Registered new tserver with Master: a270733bce914c52859a7a1cc8ae5354 (127.2.170.195:43435)
I20251209 03:02:36.759227 3215 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.759330 3215 heartbeater.cc:507] Master 127.2.170.252:37313 requested a full tablet report, sending...
I20251209 03:02:36.759516 2893 ts_manager.cc:194] Registered new tserver with Master: a270733bce914c52859a7a1cc8ae5354 (127.2.170.195:43435)
I20251209 03:02:36.760298 3217 heartbeater.cc:344] Connected to a master server at 127.2.170.253:42009
I20251209 03:02:36.760340 2893 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35156
I20251209 03:02:36.760344 3217 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:36.760479 3217 heartbeater.cc:507] Master 127.2.170.253:42009 requested a full tablet report, sending...
I20251209 03:02:36.760882 2827 ts_manager.cc:194] Registered new tserver with Master: a270733bce914c52859a7a1cc8ae5354 (127.2.170.195:43435)
I20251209 03:02:36.761579 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006463856s
I20251209 03:02:37.637792 2959 catalog_manager.cc:1269] Loaded cluster ID: 6df2b828b8e94224978fc0f488591165
I20251209 03:02:37.637867 2959 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9: loading cluster ID for follower catalog manager: success
I20251209 03:02:37.638590 2959 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9: acquiring CA information for follower catalog manager: success
I20251209 03:02:37.638780 2959 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251209 03:02:37.652027 2972 catalog_manager.cc:1269] Loaded cluster ID: 6df2b828b8e94224978fc0f488591165
I20251209 03:02:37.652073 2972 catalog_manager.cc:1562] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3: loading cluster ID for follower catalog manager: success
I20251209 03:02:37.652776 2972 catalog_manager.cc:1584] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3: acquiring CA information for follower catalog manager: success
I20251209 03:02:37.652905 2972 catalog_manager.cc:1612] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251209 03:02:37.677588 2893 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35138
I20251209 03:02:37.696914 2893 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35154
I20251209 03:02:37.760960 3215 heartbeater.cc:499] Master 127.2.170.252:37313 was elected leader, sending a full tablet report...
I20251209 03:02:38.615147 2944 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:02:38.678102 3045 heartbeater.cc:499] Master 127.2.170.252:37313 was elected leader, sending a full tablet report...
I20251209 03:02:38.697450 3124 heartbeater.cc:499] Master 127.2.170.252:37313 was elected leader, sending a full tablet report...
I20251209 03:02:38.811686 2731 test_util.cc:276] Using random seed: 620286498
I20251209 03:02:38.823161 2893 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35170:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251209 03:02:38.829854 3179 tablet_service.cc:1505] Processing CreateTablet for tablet 64b3a16402c94bee949c357879c946db (DEFAULT_TABLE table=test-workload [id=576e913351d948e7bec1475d6abb8d4f]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:02:38.830057 3179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64b3a16402c94bee949c357879c946db. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:38.830263 3009 tablet_service.cc:1505] Processing CreateTablet for tablet 06b1acd664864b5faa1ff800a798e0a5 (DEFAULT_TABLE table=test-workload [id=576e913351d948e7bec1475d6abb8d4f]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:02:38.830369 3009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b1acd664864b5faa1ff800a798e0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:38.831226 3250 tablet_bootstrap.cc:492] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354: Bootstrap starting.
I20251209 03:02:38.831284 3008 tablet_service.cc:1505] Processing CreateTablet for tablet a25a94aa28ab4398ad04549647d148e0 (DEFAULT_TABLE table=test-workload [id=576e913351d948e7bec1475d6abb8d4f]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:02:38.831377 3008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a25a94aa28ab4398ad04549647d148e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:38.831662 3250 tablet_bootstrap.cc:654] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:38.831914 3251 tablet_bootstrap.cc:492] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9: Bootstrap starting.
I20251209 03:02:38.832160 3251 tablet_bootstrap.cc:654] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:38.832181 3250 tablet_bootstrap.cc:492] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354: No bootstrap required, opened a new log
I20251209 03:02:38.832216 3250 ts_tablet_manager.cc:1403] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.832368 3250 raft_consensus.cc:359] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a270733bce914c52859a7a1cc8ae5354" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43435 } }
I20251209 03:02:38.832417 3250 raft_consensus.cc:385] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:38.832442 3250 raft_consensus.cc:740] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a270733bce914c52859a7a1cc8ae5354, State: Initialized, Role: FOLLOWER
I20251209 03:02:38.832485 3250 consensus_queue.cc:260] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a270733bce914c52859a7a1cc8ae5354" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43435 } }
I20251209 03:02:38.832530 3250 raft_consensus.cc:399] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:38.832551 3250 raft_consensus.cc:493] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:38.832567 3250 raft_consensus.cc:3060] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:38.832643 3088 tablet_service.cc:1505] Processing CreateTablet for tablet 641412f709a143758895b23dd7f28a15 (DEFAULT_TABLE table=test-workload [id=576e913351d948e7bec1475d6abb8d4f]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:02:38.832747 3088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 641412f709a143758895b23dd7f28a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:38.832818 3251 tablet_bootstrap.cc:492] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9: No bootstrap required, opened a new log
I20251209 03:02:38.832854 3251 ts_tablet_manager.cc:1403] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.832978 3251 raft_consensus.cc:359] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } }
I20251209 03:02:38.833019 3251 raft_consensus.cc:385] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:38.833034 3251 raft_consensus.cc:740] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7b2d74b8384bd3bbd73c9792491cb9, State: Initialized, Role: FOLLOWER
I20251209 03:02:38.833076 3251 consensus_queue.cc:260] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } }
I20251209 03:02:38.833106 3251 raft_consensus.cc:399] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:38.833123 3251 raft_consensus.cc:493] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:38.833140 3251 raft_consensus.cc:3060] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:38.833161 3250 raft_consensus.cc:515] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a270733bce914c52859a7a1cc8ae5354" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43435 } }
I20251209 03:02:38.833204 3250 leader_election.cc:304] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a270733bce914c52859a7a1cc8ae5354; no voters:
I20251209 03:02:38.833304 3250 leader_election.cc:290] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:38.833353 3254 raft_consensus.cc:2804] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:38.833416 3250 ts_tablet_manager.cc:1434] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.833467 3254 raft_consensus.cc:697] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 1 LEADER]: Becoming Leader. State: Replica: a270733bce914c52859a7a1cc8ae5354, State: Running, Role: LEADER
I20251209 03:02:38.833508 3254 consensus_queue.cc:237] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a270733bce914c52859a7a1cc8ae5354" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43435 } }
I20251209 03:02:38.834008 3251 raft_consensus.cc:515] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } }
I20251209 03:02:38.834054 3251 leader_election.cc:304] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d7b2d74b8384bd3bbd73c9792491cb9; no voters:
I20251209 03:02:38.834144 3251 leader_election.cc:290] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:38.834100 2893 catalog_manager.cc:5654] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 reported cstate change: term changed from 0 to 1, leader changed from <none> to a270733bce914c52859a7a1cc8ae5354 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "a270733bce914c52859a7a1cc8ae5354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a270733bce914c52859a7a1cc8ae5354" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43435 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:38.834606 3251 ts_tablet_manager.cc:1434] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:02:38.834853 3251 tablet_bootstrap.cc:492] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9: Bootstrap starting.
I20251209 03:02:38.835225 3251 tablet_bootstrap.cc:654] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:38.835325 3258 tablet_bootstrap.cc:492] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02: Bootstrap starting.
I20251209 03:02:38.835671 3258 tablet_bootstrap.cc:654] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:38.836069 3261 raft_consensus.cc:2804] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:38.836171 3261 raft_consensus.cc:697] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7b2d74b8384bd3bbd73c9792491cb9, State: Running, Role: LEADER
I20251209 03:02:38.836220 3261 consensus_queue.cc:237] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } }
I20251209 03:02:38.836277 3258 tablet_bootstrap.cc:492] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02: No bootstrap required, opened a new log
I20251209 03:02:38.836344 3258 ts_tablet_manager.cc:1403] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.836449 3258 raft_consensus.cc:359] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3c9240e20643dd8648c5cedf32ef02" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43941 } }
I20251209 03:02:38.836522 3258 raft_consensus.cc:385] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:38.836560 3258 raft_consensus.cc:740] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3c9240e20643dd8648c5cedf32ef02, State: Initialized, Role: FOLLOWER
I20251209 03:02:38.836617 3258 consensus_queue.cc:260] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3c9240e20643dd8648c5cedf32ef02" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43941 } }
I20251209 03:02:38.836685 3258 raft_consensus.cc:399] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:38.836673 2892 catalog_manager.cc:5654] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d7b2d74b8384bd3bbd73c9792491cb9 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:38.836794 3258 raft_consensus.cc:493] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:38.836820 3258 raft_consensus.cc:3060] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:38.837745 3258 raft_consensus.cc:515] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3c9240e20643dd8648c5cedf32ef02" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43941 } }
I20251209 03:02:38.838066 3251 tablet_bootstrap.cc:492] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9: No bootstrap required, opened a new log
I20251209 03:02:38.838112 3258 leader_election.cc:304] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac3c9240e20643dd8648c5cedf32ef02; no voters:
I20251209 03:02:38.838174 3251 ts_tablet_manager.cc:1403] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251209 03:02:38.838284 3258 leader_election.cc:290] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:38.838291 3251 raft_consensus.cc:359] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } }
I20251209 03:02:38.838404 3251 raft_consensus.cc:385] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:38.838487 3251 raft_consensus.cc:740] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7b2d74b8384bd3bbd73c9792491cb9, State: Initialized, Role: FOLLOWER
I20251209 03:02:38.838537 3251 consensus_queue.cc:260] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } }
I20251209 03:02:38.838582 3251 raft_consensus.cc:399] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:38.838598 3251 raft_consensus.cc:493] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:38.838609 3251 raft_consensus.cc:3060] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:38.839017 3251 raft_consensus.cc:515] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } }
I20251209 03:02:38.839071 3251 leader_election.cc:304] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d7b2d74b8384bd3bbd73c9792491cb9; no voters:
I20251209 03:02:38.839098 3251 leader_election.cc:290] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:38.839146 3261 raft_consensus.cc:2804] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:38.838416 3258 ts_tablet_manager.cc:1434] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251209 03:02:38.839238 3261 raft_consensus.cc:697] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7b2d74b8384bd3bbd73c9792491cb9, State: Running, Role: LEADER
I20251209 03:02:38.839162 3251 ts_tablet_manager.cc:1434] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.839321 3261 consensus_queue.cc:237] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } }
I20251209 03:02:38.838316 3267 raft_consensus.cc:2804] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:38.839403 3267 raft_consensus.cc:697] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 1 LEADER]: Becoming Leader. State: Replica: ac3c9240e20643dd8648c5cedf32ef02, State: Running, Role: LEADER
I20251209 03:02:38.839449 3267 consensus_queue.cc:237] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3c9240e20643dd8648c5cedf32ef02" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43941 } }
I20251209 03:02:38.839690 2892 catalog_manager.cc:5654] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d7b2d74b8384bd3bbd73c9792491cb9 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7b2d74b8384bd3bbd73c9792491cb9" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35077 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:38.840757 2892 catalog_manager.cc:5654] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac3c9240e20643dd8648c5cedf32ef02 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "ac3c9240e20643dd8648c5cedf32ef02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3c9240e20643dd8648c5cedf32ef02" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43941 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:38.842020 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:02:38.844306 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:38.844468 2731 tablet_replica.cc:333] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9: stopping tablet replica
I20251209 03:02:38.844532 2731 raft_consensus.cc:2243] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:38.844574 2731 raft_consensus.cc:2272] T a25a94aa28ab4398ad04549647d148e0 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:38.844839 2731 tablet_replica.cc:333] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9: stopping tablet replica
I20251209 03:02:38.844875 2731 raft_consensus.cc:2243] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:38.844918 2731 raft_consensus.cc:2272] T 06b1acd664864b5faa1ff800a798e0a5 P 6d7b2d74b8384bd3bbd73c9792491cb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:38.856581 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:02:38.857403 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:02:38.859759 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:38.859869 2731 tablet_replica.cc:333] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02: stopping tablet replica
I20251209 03:02:38.859905 2731 raft_consensus.cc:2243] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:38.859937 2731 pending_rounds.cc:70] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02: Trying to abort 1 pending ops.
I20251209 03:02:38.859961 2731 pending_rounds.cc:77] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7230461373806575616 op_type: NO_OP noop_request { }
I20251209 03:02:38.860004 2731 raft_consensus.cc:2889] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251209 03:02:38.860018 2731 raft_consensus.cc:2272] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251209 03:02:38.861801 3270 consensus_queue.cc:1175] T 641412f709a143758895b23dd7f28a15 P ac3c9240e20643dd8648c5cedf32ef02 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ac3c9240e20643dd8648c5cedf32ef02" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251209 03:02:38.873123 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:02:38.873854 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:02:38.876014 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:38.876204 2731 tablet_replica.cc:333] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354: stopping tablet replica
I20251209 03:02:38.876252 2731 raft_consensus.cc:2243] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:38.876293 2731 raft_consensus.cc:2272] T 64b3a16402c94bee949c357879c946db P a270733bce914c52859a7a1cc8ae5354 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:38.887775 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:02:38.888388 2731 master.cc:561] Master@127.2.170.254:33821 shutting down...
I20251209 03:02:38.890110 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:38.890168 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:38.890189 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 00f1195cb8c548618455d7affd4dedd9: stopping tablet replica
I20251209 03:02:38.901639 2731 master.cc:583] Master@127.2.170.254:33821 shutdown complete.
I20251209 03:02:38.902580 2731 master.cc:561] Master@127.2.170.253:42009 shutting down...
I20251209 03:02:38.904549 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:38.904613 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:38.904630 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P dd3f55b6384b451f9b28474bfd2c1cb3: stopping tablet replica
I20251209 03:02:38.916078 2731 master.cc:583] Master@127.2.170.253:42009 shutdown complete.
I20251209 03:02:38.916934 2731 master.cc:561] Master@127.2.170.252:37313 shutting down...
I20251209 03:02:38.918893 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:38.919006 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:38.919036 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7f8c9e8f2a5c49038e3b2e46994f5f77: stopping tablet replica
I20251209 03:02:38.920686 2731 master.cc:583] Master@127.2.170.252:37313 shutdown complete.
[ OK ] AutoRebalancerTest.OnlyLeaderDoesAutoRebalancing (2383 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn
I20251209 03:02:38.925242 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:43279
I20251209 03:02:38.925377 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:38.926406 3271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:38.926448 3272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:38.926501 3274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:38.926707 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:38.926762 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:38.926784 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:38.926792 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249358926793 us; error 0 us; skew 500 ppm
I20251209 03:02:38.927245 2731 webserver.cc:492] Webserver started at http://127.2.170.254:46665/ using document root <none> and password file <none>
I20251209 03:02:38.927304 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:38.927330 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:38.927361 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:38.927544 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "f187b6dc264346ae9500ed084d70135d"
format_stamp: "Formatted at 2025-12-09 03:02:38 on dist-test-slave-4cb0"
I20251209 03:02:38.928232 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.928584 3279 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.928692 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.928732 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "f187b6dc264346ae9500ed084d70135d"
format_stamp: "Formatted at 2025-12-09 03:02:38 on dist-test-slave-4cb0"
I20251209 03:02:38.928769 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:38.939254 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:38.939383 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:38.941561 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:43279
I20251209 03:02:38.944054 3331 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:43279 every 8 connection(s)
I20251209 03:02:38.944203 3332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:38.945106 3332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d: Bootstrap starting.
I20251209 03:02:38.945345 3332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:38.945792 3332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d: No bootstrap required, opened a new log
I20251209 03:02:38.945904 3332 raft_consensus.cc:359] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f187b6dc264346ae9500ed084d70135d" member_type: VOTER }
I20251209 03:02:38.945947 3332 raft_consensus.cc:385] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:38.945966 3332 raft_consensus.cc:740] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f187b6dc264346ae9500ed084d70135d, State: Initialized, Role: FOLLOWER
I20251209 03:02:38.946007 3332 consensus_queue.cc:260] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f187b6dc264346ae9500ed084d70135d" member_type: VOTER }
I20251209 03:02:38.946041 3332 raft_consensus.cc:399] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:38.946060 3332 raft_consensus.cc:493] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:38.946081 3332 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:38.946496 3332 raft_consensus.cc:515] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f187b6dc264346ae9500ed084d70135d" member_type: VOTER }
I20251209 03:02:38.946547 3332 leader_election.cc:304] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f187b6dc264346ae9500ed084d70135d; no voters:
I20251209 03:02:38.946635 3332 leader_election.cc:290] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:38.946677 3335 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:38.946769 3332 sys_catalog.cc:565] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:02:38.946828 3335 raft_consensus.cc:697] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 1 LEADER]: Becoming Leader. State: Replica: f187b6dc264346ae9500ed084d70135d, State: Running, Role: LEADER
I20251209 03:02:38.946880 3335 consensus_queue.cc:237] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f187b6dc264346ae9500ed084d70135d" member_type: VOTER }
I20251209 03:02:38.947062 3336 sys_catalog.cc:455] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f187b6dc264346ae9500ed084d70135d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f187b6dc264346ae9500ed084d70135d" member_type: VOTER } }
I20251209 03:02:38.947075 3337 sys_catalog.cc:455] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f187b6dc264346ae9500ed084d70135d. Latest consensus state: current_term: 1 leader_uuid: "f187b6dc264346ae9500ed084d70135d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f187b6dc264346ae9500ed084d70135d" member_type: VOTER } }
I20251209 03:02:38.947131 3336 sys_catalog.cc:458] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:38.947142 3337 sys_catalog.cc:458] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:38.947850 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20251209 03:02:38.947937 3351 catalog_manager.cc:1568] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251209 03:02:38.947975 3351 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251209 03:02:38.948032 3341 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:02:38.948153 3341 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:02:38.948621 3341 catalog_manager.cc:1357] Generated new cluster ID: 7cf8e4b84eff4f41b8746be2a2b6fe9f
I20251209 03:02:38.948663 3341 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:02:38.955921 3341 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:02:38.956287 3341 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:02:38.961503 3341 catalog_manager.cc:6027] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d: Generated new TSK 0
I20251209 03:02:38.961576 3341 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:02:38.963352 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:38.964511 3354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:38.964498 3356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:38.964685 3353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:38.964834 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:38.964917 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:38.964939 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:38.964951 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249358964951 us; error 0 us; skew 500 ppm
I20251209 03:02:38.965428 2731 webserver.cc:492] Webserver started at http://127.2.170.193:33869/ using document root <none> and password file <none>
I20251209 03:02:38.965481 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:38.965509 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:38.965538 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:38.965780 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "9ea557de30c24f65b427f0908c290a80"
format_stamp: "Formatted at 2025-12-09 03:02:38 on dist-test-slave-4cb0"
I20251209 03:02:38.966480 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.966852 3361 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.966984 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.967028 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "9ea557de30c24f65b427f0908c290a80"
format_stamp: "Formatted at 2025-12-09 03:02:38 on dist-test-slave-4cb0"
I20251209 03:02:38.967067 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:38.973125 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:38.973280 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:38.973387 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:38.973539 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:38.973564 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.973588 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:38.973613 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.976047 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:46489
I20251209 03:02:38.976079 3424 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:46489 every 8 connection(s)
I20251209 03:02:38.976329 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:38.977262 3429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:38.977236 3430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:38.977522 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:38.977543 3432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:38.977725 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:38.977762 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:38.977778 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249358977779 us; error 0 us; skew 500 ppm
I20251209 03:02:38.978264 2731 webserver.cc:492] Webserver started at http://127.2.170.194:33895/ using document root <none> and password file <none>
I20251209 03:02:38.978336 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:38.978381 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:38.978422 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:38.978615 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1"
format_stamp: "Formatted at 2025-12-09 03:02:38 on dist-test-slave-4cb0"
I20251209 03:02:38.979481 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.979933 3425 heartbeater.cc:344] Connected to a master server at 127.2.170.254:43279
I20251209 03:02:38.979987 3437 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.979988 3425 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:38.980114 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.980130 3425 heartbeater.cc:507] Master 127.2.170.254:43279 requested a full tablet report, sending...
I20251209 03:02:38.980178 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1"
format_stamp: "Formatted at 2025-12-09 03:02:38 on dist-test-slave-4cb0"
I20251209 03:02:38.980255 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:38.980350 3296 ts_manager.cc:194] Registered new tserver with Master: 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489)
I20251209 03:02:38.980940 3296 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44818
I20251209 03:02:38.984062 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:38.984236 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:38.984354 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:38.984525 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:38.984553 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.984581 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:38.984608 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.987529 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:43185
I20251209 03:02:38.987581 3500 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:43185 every 8 connection(s)
I20251209 03:02:38.987998 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:38.989006 3504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:38.989045 3505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:38.989464 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:38.989468 3507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:38.989686 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:38.989725 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:38.989751 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249358989751 us; error 0 us; skew 500 ppm
I20251209 03:02:38.990257 2731 webserver.cc:492] Webserver started at http://127.2.170.195:40809/ using document root <none> and password file <none>
I20251209 03:02:38.990342 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:38.990410 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:38.990458 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:38.990715 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "9533c58a138b4d0e9137093637002f32"
format_stamp: "Formatted at 2025-12-09 03:02:38 on dist-test-slave-4cb0"
I20251209 03:02:38.990810 3501 heartbeater.cc:344] Connected to a master server at 127.2.170.254:43279
I20251209 03:02:38.990867 3501 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:38.990959 3501 heartbeater.cc:507] Master 127.2.170.254:43279 requested a full tablet report, sending...
I20251209 03:02:38.991147 3296 ts_manager.cc:194] Registered new tserver with Master: 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185)
I20251209 03:02:38.991463 3296 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44832
I20251209 03:02:38.991593 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:38.992003 3512 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.992115 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.992153 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "9533c58a138b4d0e9137093637002f32"
format_stamp: "Formatted at 2025-12-09 03:02:38 on dist-test-slave-4cb0"
I20251209 03:02:38.992192 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:38.998513 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:38.998703 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:38.998829 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:38.998991 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:38.999017 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:38.999040 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:38.999064 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:39.002079 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:40363
I20251209 03:02:39.002125 3575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:40363 every 8 connection(s)
I20251209 03:02:39.004617 3576 heartbeater.cc:344] Connected to a master server at 127.2.170.254:43279
I20251209 03:02:39.004665 3576 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:39.004745 3576 heartbeater.cc:507] Master 127.2.170.254:43279 requested a full tablet report, sending...
I20251209 03:02:39.004935 3296 ts_manager.cc:194] Registered new tserver with Master: 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:39.005251 3296 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44846
I20251209 03:02:39.005509 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003289798s
I20251209 03:02:39.981360 3425 heartbeater.cc:499] Master 127.2.170.254:43279 was elected leader, sending a full tablet report...
I20251209 03:02:39.991897 3501 heartbeater.cc:499] Master 127.2.170.254:43279 was elected leader, sending a full tablet report...
I20251209 03:02:40.005692 3576 heartbeater.cc:499] Master 127.2.170.254:43279 was elected leader, sending a full tablet report...
I20251209 03:02:40.030274 2731 test_util.cc:276] Using random seed: 621505087
I20251209 03:02:40.034905 3296 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44848:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o""\004\001\000\377\377\377\017\004\001\000\376\377\377\037\004\001\000\375\377\377/\004\001\000\374\377\377?\004\001\000\373\377\377O\004\001\000\372\377\377_\004\001\000\371\377\377o"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:02:40.035063 3296 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:02:40.039857 3540 tablet_service.cc:1505] Processing CreateTablet for tablet da77480623e44fd9ad9c75acdfa7939c (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251209 03:02:40.040004 3540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da77480623e44fd9ad9c75acdfa7939c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.040210 3539 tablet_service.cc:1505] Processing CreateTablet for tablet e41c5d567cf643fbbe0c2089d57d0dfa (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251209 03:02:40.040314 3539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41c5d567cf643fbbe0c2089d57d0dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.040351 3464 tablet_service.cc:1505] Processing CreateTablet for tablet e41c5d567cf643fbbe0c2089d57d0dfa (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251209 03:02:40.040442 3464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41c5d567cf643fbbe0c2089d57d0dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.041337 3538 tablet_service.cc:1505] Processing CreateTablet for tablet 351addfed22a4cfca1193384c0e2dcb0 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251209 03:02:40.041420 3538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351addfed22a4cfca1193384c0e2dcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.041474 3596 tablet_bootstrap.cc:492] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32: Bootstrap starting.
I20251209 03:02:40.041749 3597 tablet_bootstrap.cc:492] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1: Bootstrap starting.
I20251209 03:02:40.041779 3462 tablet_service.cc:1505] Processing CreateTablet for tablet af124c3de2584f59938f5c799d8b9a63 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251209 03:02:40.041852 3462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af124c3de2584f59938f5c799d8b9a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.042145 3596 tablet_bootstrap.cc:654] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.042162 3597 tablet_bootstrap.cc:654] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.042619 3537 tablet_service.cc:1505] Processing CreateTablet for tablet af124c3de2584f59938f5c799d8b9a63 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251209 03:02:40.042698 3537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af124c3de2584f59938f5c799d8b9a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.042855 3461 tablet_service.cc:1505] Processing CreateTablet for tablet bf09779c2d284f9baa2d1f51a18641bd (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251209 03:02:40.042917 3461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf09779c2d284f9baa2d1f51a18641bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.043527 3536 tablet_service.cc:1505] Processing CreateTablet for tablet bf09779c2d284f9baa2d1f51a18641bd (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251209 03:02:40.043599 3536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf09779c2d284f9baa2d1f51a18641bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.043709 3460 tablet_service.cc:1505] Processing CreateTablet for tablet 488d620d01094c2888abd1dd3bce3313 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251209 03:02:40.043769 3460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488d620d01094c2888abd1dd3bce3313. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.044385 3535 tablet_service.cc:1505] Processing CreateTablet for tablet 488d620d01094c2888abd1dd3bce3313 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251209 03:02:40.044457 3535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488d620d01094c2888abd1dd3bce3313. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.044575 3459 tablet_service.cc:1505] Processing CreateTablet for tablet a3bf844993c44f5e90c8e2c2388f89b7 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251209 03:02:40.044656 3459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3bf844993c44f5e90c8e2c2388f89b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.045397 3534 tablet_service.cc:1505] Processing CreateTablet for tablet a3bf844993c44f5e90c8e2c2388f89b7 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251209 03:02:40.045471 3534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3bf844993c44f5e90c8e2c2388f89b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.045612 3458 tablet_service.cc:1505] Processing CreateTablet for tablet ad233cf84ae24f6499c25e0fa6da860b (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251209 03:02:40.045712 3458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad233cf84ae24f6499c25e0fa6da860b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.045795 3533 tablet_service.cc:1505] Processing CreateTablet for tablet ad233cf84ae24f6499c25e0fa6da860b (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251209 03:02:40.045889 3533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad233cf84ae24f6499c25e0fa6da860b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.046681 3389 tablet_service.cc:1505] Processing CreateTablet for tablet da77480623e44fd9ad9c75acdfa7939c (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251209 03:02:40.046764 3383 tablet_service.cc:1505] Processing CreateTablet for tablet a3bf844993c44f5e90c8e2c2388f89b7 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1879048185
I20251209 03:02:40.046789 3389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da77480623e44fd9ad9c75acdfa7939c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.046687 3388 tablet_service.cc:1505] Processing CreateTablet for tablet e41c5d567cf643fbbe0c2089d57d0dfa (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 268435455 <= VALUES < 536870910
I20251209 03:02:40.046852 3383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3bf844993c44f5e90c8e2c2388f89b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.040325 3465 tablet_service.cc:1505] Processing CreateTablet for tablet da77480623e44fd9ad9c75acdfa7939c (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION VALUES < 268435455
I20251209 03:02:40.046986 3384 tablet_service.cc:1505] Processing CreateTablet for tablet 488d620d01094c2888abd1dd3bce3313 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1342177275 <= VALUES < 1610612730
I20251209 03:02:40.047005 3465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da77480623e44fd9ad9c75acdfa7939c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.047035 3388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41c5d567cf643fbbe0c2089d57d0dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.047127 3382 tablet_service.cc:1505] Processing CreateTablet for tablet ad233cf84ae24f6499c25e0fa6da860b (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1879048185 <= VALUES
I20251209 03:02:40.047202 3382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad233cf84ae24f6499c25e0fa6da860b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.048246 3600 tablet_bootstrap.cc:492] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80: Bootstrap starting.
I20251209 03:02:40.041749 3463 tablet_service.cc:1505] Processing CreateTablet for tablet 351addfed22a4cfca1193384c0e2dcb0 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251209 03:02:40.048411 3463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351addfed22a4cfca1193384c0e2dcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.048884 3600 tablet_bootstrap.cc:654] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.049085 3596 tablet_bootstrap.cc:492] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32: No bootstrap required, opened a new log
I20251209 03:02:40.049122 3596 ts_tablet_manager.cc:1403] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20251209 03:02:40.049268 3597 tablet_bootstrap.cc:492] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1: No bootstrap required, opened a new log
I20251209 03:02:40.049254 3596 raft_consensus.cc:359] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.049326 3597 ts_tablet_manager.cc:1403] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20251209 03:02:40.049340 3596 raft_consensus.cc:385] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.049360 3596 raft_consensus.cc:740] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.046802 3387 tablet_service.cc:1505] Processing CreateTablet for tablet 351addfed22a4cfca1193384c0e2dcb0 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 805306365
I20251209 03:02:40.049397 3596 consensus_queue.cc:260] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.049480 3387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351addfed22a4cfca1193384c0e2dcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.049463 3597 raft_consensus.cc:359] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.049521 3597 raft_consensus.cc:385] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.049540 3597 raft_consensus.cc:740] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.049551 3596 ts_tablet_manager.cc:1434] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251209 03:02:40.049568 3597 consensus_queue.cc:260] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.049599 3596 tablet_bootstrap.cc:492] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32: Bootstrap starting.
I20251209 03:02:40.049736 3597 ts_tablet_manager.cc:1434] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251209 03:02:40.049806 3597 tablet_bootstrap.cc:492] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Bootstrap starting.
I20251209 03:02:40.050000 3596 tablet_bootstrap.cc:654] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.050187 3597 tablet_bootstrap.cc:654] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.050514 3596 tablet_bootstrap.cc:492] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32: No bootstrap required, opened a new log
I20251209 03:02:40.050551 3596 ts_tablet_manager.cc:1403] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:40.046904 3385 tablet_service.cc:1505] Processing CreateTablet for tablet bf09779c2d284f9baa2d1f51a18641bd (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1342177275
I20251209 03:02:40.050658 3385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf09779c2d284f9baa2d1f51a18641bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.050683 3596 raft_consensus.cc:359] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.050712 3597 tablet_bootstrap.cc:492] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1: No bootstrap required, opened a new log
I20251209 03:02:40.050750 3596 raft_consensus.cc:385] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.050771 3597 ts_tablet_manager.cc:1403] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.050772 3596 raft_consensus.cc:740] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.050848 3596 consensus_queue.cc:260] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.050925 3596 ts_tablet_manager.cc:1434] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.050962 3597 raft_consensus.cc:359] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.051028 3597 raft_consensus.cc:385] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.051045 3597 raft_consensus.cc:740] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.051079 3597 consensus_queue.cc:260] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.050967 3596 tablet_bootstrap.cc:492] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32: Bootstrap starting.
I20251209 03:02:40.051203 3597 ts_tablet_manager.cc:1434] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.051252 3597 tablet_bootstrap.cc:492] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1: Bootstrap starting.
I20251209 03:02:40.051506 3596 tablet_bootstrap.cc:654] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.051611 3597 tablet_bootstrap.cc:654] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.046838 3386 tablet_service.cc:1505] Processing CreateTablet for tablet af124c3de2584f59938f5c799d8b9a63 (DEFAULT_TABLE table=test-workload [id=5d706ad75096499daf0d5025c550d619]), partition=RANGE (key) PARTITION 805306365 <= VALUES < 1073741820
I20251209 03:02:40.051858 3386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af124c3de2584f59938f5c799d8b9a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.052073 3596 tablet_bootstrap.cc:492] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32: No bootstrap required, opened a new log
I20251209 03:02:40.052088 3597 tablet_bootstrap.cc:492] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1: No bootstrap required, opened a new log
I20251209 03:02:40.052109 3596 ts_tablet_manager.cc:1403] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:02:40.052122 3597 ts_tablet_manager.cc:1403] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.052206 3596 raft_consensus.cc:359] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.052241 3597 raft_consensus.cc:359] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.052246 3596 raft_consensus.cc:385] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.052307 3597 raft_consensus.cc:385] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.052311 3596 raft_consensus.cc:740] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.052338 3597 raft_consensus.cc:740] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.052394 3596 consensus_queue.cc:260] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.052477 3596 ts_tablet_manager.cc:1434] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.052433 3597 consensus_queue.cc:260] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.052528 3596 tablet_bootstrap.cc:492] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32: Bootstrap starting.
I20251209 03:02:40.052537 3597 ts_tablet_manager.cc:1434] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.052578 3597 tablet_bootstrap.cc:492] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Bootstrap starting.
I20251209 03:02:40.052912 3384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488d620d01094c2888abd1dd3bce3313. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:40.052961 3597 tablet_bootstrap.cc:654] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.052961 3596 tablet_bootstrap.cc:654] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.053519 3596 tablet_bootstrap.cc:492] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32: No bootstrap required, opened a new log
I20251209 03:02:40.053530 3597 tablet_bootstrap.cc:492] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1: No bootstrap required, opened a new log
I20251209 03:02:40.053570 3596 ts_tablet_manager.cc:1403] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:02:40.053579 3597 ts_tablet_manager.cc:1403] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.053714 3597 raft_consensus.cc:359] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.053714 3596 raft_consensus.cc:359] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.053790 3597 raft_consensus.cc:385] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.053795 3596 raft_consensus.cc:385] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.053819 3597 raft_consensus.cc:740] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.053843 3596 raft_consensus.cc:740] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.053884 3597 consensus_queue.cc:260] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.053896 3596 consensus_queue.cc:260] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.053963 3597 ts_tablet_manager.cc:1434] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.054005 3597 tablet_bootstrap.cc:492] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Bootstrap starting.
I20251209 03:02:40.054296 3596 ts_tablet_manager.cc:1434] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:40.054366 3597 tablet_bootstrap.cc:654] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.054382 3596 tablet_bootstrap.cc:492] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32: Bootstrap starting.
I20251209 03:02:40.054886 3597 tablet_bootstrap.cc:492] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1: No bootstrap required, opened a new log
I20251209 03:02:40.054896 3596 tablet_bootstrap.cc:654] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.054939 3597 ts_tablet_manager.cc:1403] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:40.055181 3597 raft_consensus.cc:359] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.055233 3597 raft_consensus.cc:385] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.055253 3597 raft_consensus.cc:740] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.055289 3597 consensus_queue.cc:260] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.055387 3597 ts_tablet_manager.cc:1434] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.055403 3600 tablet_bootstrap.cc:492] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80: No bootstrap required, opened a new log
I20251209 03:02:40.055433 3597 tablet_bootstrap.cc:492] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1: Bootstrap starting.
I20251209 03:02:40.055444 3600 ts_tablet_manager.cc:1403] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20251209 03:02:40.055495 3596 tablet_bootstrap.cc:492] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32: No bootstrap required, opened a new log
I20251209 03:02:40.055527 3596 ts_tablet_manager.cc:1403] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:40.055583 3600 raft_consensus.cc:359] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.055646 3600 raft_consensus.cc:385] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.055687 3600 raft_consensus.cc:740] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.055651 3596 raft_consensus.cc:359] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.055748 3596 raft_consensus.cc:385] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.055752 3600 consensus_queue.cc:260] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.055805 3597 tablet_bootstrap.cc:654] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.055778 3596 raft_consensus.cc:740] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.055882 3596 consensus_queue.cc:260] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.055943 3600 ts_tablet_manager.cc:1434] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:02:40.055959 3596 ts_tablet_manager.cc:1434] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.056010 3600 tablet_bootstrap.cc:492] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80: Bootstrap starting.
I20251209 03:02:40.056016 3596 tablet_bootstrap.cc:492] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32: Bootstrap starting.
I20251209 03:02:40.056387 3600 tablet_bootstrap.cc:654] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.056677 3596 tablet_bootstrap.cc:654] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.056808 3600 tablet_bootstrap.cc:492] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80: No bootstrap required, opened a new log
I20251209 03:02:40.056835 3600 ts_tablet_manager.cc:1403] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:40.056932 3600 raft_consensus.cc:359] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.056979 3600 raft_consensus.cc:385] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.056996 3600 raft_consensus.cc:740] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.057034 3600 consensus_queue.cc:260] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.057147 3596 tablet_bootstrap.cc:492] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32: No bootstrap required, opened a new log
I20251209 03:02:40.057168 3600 ts_tablet_manager.cc:1434] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.057191 3596 ts_tablet_manager.cc:1403] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:40.057219 3600 tablet_bootstrap.cc:492] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80: Bootstrap starting.
I20251209 03:02:40.057349 3596 raft_consensus.cc:359] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.057395 3596 raft_consensus.cc:385] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.057413 3596 raft_consensus.cc:740] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.057446 3596 consensus_queue.cc:260] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.057525 3596 ts_tablet_manager.cc:1434] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.057569 3596 tablet_bootstrap.cc:492] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32: Bootstrap starting.
I20251209 03:02:40.057577 3600 tablet_bootstrap.cc:654] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.057969 3596 tablet_bootstrap.cc:654] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.058192 3600 tablet_bootstrap.cc:492] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80: No bootstrap required, opened a new log
I20251209 03:02:40.058230 3600 ts_tablet_manager.cc:1403] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.058347 3600 raft_consensus.cc:359] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.058404 3600 raft_consensus.cc:385] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.058424 3600 raft_consensus.cc:740] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.058465 3600 consensus_queue.cc:260] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.058483 3596 tablet_bootstrap.cc:492] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32: No bootstrap required, opened a new log
I20251209 03:02:40.058552 3596 ts_tablet_manager.cc:1403] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.058589 3600 ts_tablet_manager.cc:1434] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.058638 3600 tablet_bootstrap.cc:492] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80: Bootstrap starting.
I20251209 03:02:40.058650 3597 tablet_bootstrap.cc:492] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1: No bootstrap required, opened a new log
I20251209 03:02:40.058681 3597 ts_tablet_manager.cc:1403] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251209 03:02:40.058694 3596 raft_consensus.cc:359] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.058763 3596 raft_consensus.cc:385] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.058782 3596 raft_consensus.cc:740] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.058817 3596 consensus_queue.cc:260] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.058830 3597 raft_consensus.cc:359] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.058872 3597 raft_consensus.cc:385] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.058889 3597 raft_consensus.cc:740] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.058894 3596 ts_tablet_manager.cc:1434] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.058928 3597 consensus_queue.cc:260] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.058951 3596 tablet_bootstrap.cc:492] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32: Bootstrap starting.
I20251209 03:02:40.059006 3597 ts_tablet_manager.cc:1434] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.059070 3597 tablet_bootstrap.cc:492] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1: Bootstrap starting.
I20251209 03:02:40.059172 3600 tablet_bootstrap.cc:654] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.059324 3596 tablet_bootstrap.cc:654] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.059440 3597 tablet_bootstrap.cc:654] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.059867 3596 tablet_bootstrap.cc:492] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32: No bootstrap required, opened a new log
I20251209 03:02:40.059916 3596 ts_tablet_manager.cc:1403] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.060032 3597 tablet_bootstrap.cc:492] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1: No bootstrap required, opened a new log
I20251209 03:02:40.060094 3597 ts_tablet_manager.cc:1403] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:40.060032 3596 raft_consensus.cc:359] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.060124 3596 raft_consensus.cc:385] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.060144 3596 raft_consensus.cc:740] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.060185 3600 tablet_bootstrap.cc:492] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80: No bootstrap required, opened a new log
I20251209 03:02:40.060174 3596 consensus_queue.cc:260] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.060218 3600 ts_tablet_manager.cc:1403] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251209 03:02:40.060214 3597 raft_consensus.cc:359] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.060250 3596 ts_tablet_manager.cc:1434] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.060272 3597 raft_consensus.cc:385] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.060297 3597 raft_consensus.cc:740] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.060333 3597 consensus_queue.cc:260] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.060357 3600 raft_consensus.cc:359] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.060410 3597 ts_tablet_manager.cc:1434] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.060464 3597 tablet_bootstrap.cc:492] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Bootstrap starting.
I20251209 03:02:40.060807 3597 tablet_bootstrap.cc:654] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.061079 3600 raft_consensus.cc:385] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.061108 3600 raft_consensus.cc:740] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.061139 3600 consensus_queue.cc:260] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.061235 3600 ts_tablet_manager.cc:1434] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:02:40.061318 3600 tablet_bootstrap.cc:492] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80: Bootstrap starting.
I20251209 03:02:40.061342 3597 tablet_bootstrap.cc:492] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1: No bootstrap required, opened a new log
I20251209 03:02:40.061388 3597 ts_tablet_manager.cc:1403] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:02:40.061530 3597 raft_consensus.cc:359] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.061582 3597 raft_consensus.cc:385] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.061601 3597 raft_consensus.cc:740] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.061658 3600 tablet_bootstrap.cc:654] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.061645 3597 consensus_queue.cc:260] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.061757 3597 ts_tablet_manager.cc:1434] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.062252 3600 tablet_bootstrap.cc:492] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80: No bootstrap required, opened a new log
I20251209 03:02:40.062292 3600 ts_tablet_manager.cc:1403] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.062410 3600 raft_consensus.cc:359] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.062460 3600 raft_consensus.cc:385] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.062479 3600 raft_consensus.cc:740] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.062525 3600 consensus_queue.cc:260] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.062615 3600 ts_tablet_manager.cc:1434] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.062666 3600 tablet_bootstrap.cc:492] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80: Bootstrap starting.
I20251209 03:02:40.062994 3600 tablet_bootstrap.cc:654] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.063674 3600 tablet_bootstrap.cc:492] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80: No bootstrap required, opened a new log
I20251209 03:02:40.063714 3600 ts_tablet_manager.cc:1403] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.063834 3600 raft_consensus.cc:359] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.063875 3600 raft_consensus.cc:385] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.063892 3600 raft_consensus.cc:740] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.063933 3600 consensus_queue.cc:260] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.064018 3600 ts_tablet_manager.cc:1434] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.064092 3600 tablet_bootstrap.cc:492] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80: Bootstrap starting.
I20251209 03:02:40.064445 3600 tablet_bootstrap.cc:654] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.064936 3600 tablet_bootstrap.cc:492] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80: No bootstrap required, opened a new log
I20251209 03:02:40.064992 3600 ts_tablet_manager.cc:1403] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.065116 3600 raft_consensus.cc:359] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.065160 3600 raft_consensus.cc:385] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.065171 3600 raft_consensus.cc:740] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.065212 3600 consensus_queue.cc:260] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.065286 3600 ts_tablet_manager.cc:1434] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.065339 3600 tablet_bootstrap.cc:492] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80: Bootstrap starting.
I20251209 03:02:40.065698 3600 tablet_bootstrap.cc:654] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:40.066226 3600 tablet_bootstrap.cc:492] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80: No bootstrap required, opened a new log
I20251209 03:02:40.066262 3600 ts_tablet_manager.cc:1403] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:40.066349 3600 raft_consensus.cc:359] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.066386 3600 raft_consensus.cc:385] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:40.066398 3600 raft_consensus.cc:740] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Initialized, Role: FOLLOWER
I20251209 03:02:40.066428 3600 consensus_queue.cc:260] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.066519 3600 ts_tablet_manager.cc:1434] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.068460 3613 raft_consensus.cc:493] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:40.068509 3613 raft_consensus.cc:515] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.068655 3613 leader_election.cc:290] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.071225 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e41c5d567cf643fbbe0c2089d57d0dfa" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32" is_pre_election: true
I20251209 03:02:40.071296 3550 raft_consensus.cc:2468] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 0.
I20251209 03:02:40.071337 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e41c5d567cf643fbbe0c2089d57d0dfa" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" is_pre_election: true
I20251209 03:02:40.071393 3475 raft_consensus.cc:2468] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 0.
I20251209 03:02:40.071431 3364 leader_election.cc:304] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9533c58a138b4d0e9137093637002f32, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.071501 3613 raft_consensus.cc:2804] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:40.071537 3613 raft_consensus.cc:493] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:40.071560 3613 raft_consensus.cc:3060] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.072136 3613 raft_consensus.cc:515] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.072239 3613 leader_election.cc:290] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 election: Requested vote from peers 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.072330 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e41c5d567cf643fbbe0c2089d57d0dfa" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1"
I20251209 03:02:40.072348 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e41c5d567cf643fbbe0c2089d57d0dfa" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32"
I20251209 03:02:40.072398 3475 raft_consensus.cc:3060] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.072412 3550 raft_consensus.cc:3060] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.072934 3550 raft_consensus.cc:2468] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 1.
I20251209 03:02:40.072947 3475 raft_consensus.cc:2468] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 1.
I20251209 03:02:40.073078 3364 leader_election.cc:304] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9533c58a138b4d0e9137093637002f32, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.073160 3613 raft_consensus.cc:2804] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:40.073200 3613 raft_consensus.cc:697] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 1 LEADER]: Becoming Leader. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Running, Role: LEADER
I20251209 03:02:40.073226 3613 consensus_queue.cc:237] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.073740 3296 catalog_manager.cc:5654] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ea557de30c24f65b427f0908c290a80 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "9ea557de30c24f65b427f0908c290a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: UNKNOWN } } }
I20251209 03:02:40.081956 3602 raft_consensus.cc:493] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:40.082016 3602 raft_consensus.cc:515] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.082144 3602 leader_election.cc:290] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185)
I20251209 03:02:40.085122 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf09779c2d284f9baa2d1f51a18641bd" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" is_pre_election: true
I20251209 03:02:40.085187 3475 raft_consensus.cc:2468] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 0.
I20251209 03:02:40.085170 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf09779c2d284f9baa2d1f51a18641bd" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80" is_pre_election: true
I20251209 03:02:40.085274 3399 raft_consensus.cc:2468] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 0.
I20251209 03:02:40.085412 3514 leader_election.cc:304] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9533c58a138b4d0e9137093637002f32, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.085502 3602 raft_consensus.cc:2804] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:40.085539 3602 raft_consensus.cc:493] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:40.085563 3602 raft_consensus.cc:3060] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.086088 3602 raft_consensus.cc:515] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.086177 3602 leader_election.cc:290] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 election: Requested vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185)
I20251209 03:02:40.086333 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf09779c2d284f9baa2d1f51a18641bd" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1"
I20251209 03:02:40.086313 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf09779c2d284f9baa2d1f51a18641bd" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80"
I20251209 03:02:40.086396 3475 raft_consensus.cc:3060] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.086409 3399 raft_consensus.cc:3060] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.086809 3475 raft_consensus.cc:2468] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 1.
I20251209 03:02:40.086933 3399 raft_consensus.cc:2468] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 1.
I20251209 03:02:40.086946 3514 leader_election.cc:304] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9533c58a138b4d0e9137093637002f32; no voters:
I20251209 03:02:40.087061 3602 raft_consensus.cc:2804] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:40.087167 3602 raft_consensus.cc:697] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 1 LEADER]: Becoming Leader. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Running, Role: LEADER
I20251209 03:02:40.087206 3602 consensus_queue.cc:237] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.087616 3296 catalog_manager.cc:5654] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9533c58a138b4d0e9137093637002f32 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "9533c58a138b4d0e9137093637002f32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:40.133473 3613 raft_consensus.cc:493] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:40.133565 3613 raft_consensus.cc:515] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.133731 3613 leader_election.cc:290] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.133929 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da77480623e44fd9ad9c75acdfa7939c" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" is_pre_election: true
I20251209 03:02:40.133929 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da77480623e44fd9ad9c75acdfa7939c" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32" is_pre_election: true
I20251209 03:02:40.134059 3475 raft_consensus.cc:2468] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 0.
I20251209 03:02:40.134068 3550 raft_consensus.cc:2468] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 0.
I20251209 03:02:40.134200 3363 leader_election.cc:304] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.134310 3613 raft_consensus.cc:2804] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:40.134362 3613 raft_consensus.cc:493] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:40.134377 3613 raft_consensus.cc:3060] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.134912 3613 raft_consensus.cc:515] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.135021 3613 leader_election.cc:290] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 election: Requested vote from peers 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.135157 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da77480623e44fd9ad9c75acdfa7939c" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1"
I20251209 03:02:40.135221 3475 raft_consensus.cc:3060] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.135161 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da77480623e44fd9ad9c75acdfa7939c" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32"
I20251209 03:02:40.135259 3550 raft_consensus.cc:3060] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.135715 3475 raft_consensus.cc:2468] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 1.
I20251209 03:02:40.135771 3550 raft_consensus.cc:2468] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 1.
I20251209 03:02:40.135878 3363 leader_election.cc:304] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.135957 3613 raft_consensus.cc:2804] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:40.136003 3613 raft_consensus.cc:697] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 1 LEADER]: Becoming Leader. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Running, Role: LEADER
I20251209 03:02:40.136047 3613 consensus_queue.cc:237] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.136567 3296 catalog_manager.cc:5654] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ea557de30c24f65b427f0908c290a80 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "9ea557de30c24f65b427f0908c290a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: UNKNOWN } } }
I20251209 03:02:40.170037 3602 raft_consensus.cc:493] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:40.170130 3602 raft_consensus.cc:515] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.170260 3602 leader_election.cc:290] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185)
I20251209 03:02:40.170450 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3bf844993c44f5e90c8e2c2388f89b7" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80" is_pre_election: true
I20251209 03:02:40.170518 3399 raft_consensus.cc:2468] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 0.
I20251209 03:02:40.170482 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3bf844993c44f5e90c8e2c2388f89b7" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" is_pre_election: true
I20251209 03:02:40.170567 3475 raft_consensus.cc:2468] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 0.
I20251209 03:02:40.170653 3514 leader_election.cc:304] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9533c58a138b4d0e9137093637002f32, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.170749 3602 raft_consensus.cc:2804] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:40.170794 3602 raft_consensus.cc:493] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:40.170812 3602 raft_consensus.cc:3060] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.171684 3602 raft_consensus.cc:515] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.171775 3602 leader_election.cc:290] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 election: Requested vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185)
I20251209 03:02:40.171921 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3bf844993c44f5e90c8e2c2388f89b7" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80"
I20251209 03:02:40.171962 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3bf844993c44f5e90c8e2c2388f89b7" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1"
I20251209 03:02:40.171991 3399 raft_consensus.cc:3060] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.172008 3475 raft_consensus.cc:3060] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.172464 3475 raft_consensus.cc:2468] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 1.
I20251209 03:02:40.172474 3399 raft_consensus.cc:2468] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 1.
I20251209 03:02:40.172616 3514 leader_election.cc:304] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9533c58a138b4d0e9137093637002f32; no voters:
I20251209 03:02:40.172696 3602 raft_consensus.cc:2804] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:40.172751 3602 raft_consensus.cc:697] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 1 LEADER]: Becoming Leader. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Running, Role: LEADER
I20251209 03:02:40.172798 3602 consensus_queue.cc:237] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.173312 3296 catalog_manager.cc:5654] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9533c58a138b4d0e9137093637002f32 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "9533c58a138b4d0e9137093637002f32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:40.195221 3602 raft_consensus.cc:493] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:40.195314 3602 raft_consensus.cc:515] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.195451 3602 leader_election.cc:290] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185)
I20251209 03:02:40.195662 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488d620d01094c2888abd1dd3bce3313" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80" is_pre_election: true
I20251209 03:02:40.195662 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488d620d01094c2888abd1dd3bce3313" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" is_pre_election: true
I20251209 03:02:40.195775 3399 raft_consensus.cc:2468] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 0.
I20251209 03:02:40.195782 3475 raft_consensus.cc:2468] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 0.
I20251209 03:02:40.195911 3514 leader_election.cc:304] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9533c58a138b4d0e9137093637002f32; no voters:
I20251209 03:02:40.195991 3602 raft_consensus.cc:2804] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:40.196045 3602 raft_consensus.cc:493] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:40.196065 3602 raft_consensus.cc:3060] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.196607 3602 raft_consensus.cc:515] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.196708 3602 leader_election.cc:290] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 election: Requested vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185)
I20251209 03:02:40.196842 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488d620d01094c2888abd1dd3bce3313" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1"
I20251209 03:02:40.196847 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488d620d01094c2888abd1dd3bce3313" candidate_uuid: "9533c58a138b4d0e9137093637002f32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80"
I20251209 03:02:40.196906 3475 raft_consensus.cc:3060] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.196911 3399 raft_consensus.cc:3060] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.197489 3475 raft_consensus.cc:2468] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 1.
I20251209 03:02:40.197489 3399 raft_consensus.cc:2468] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9533c58a138b4d0e9137093637002f32 in term 1.
I20251209 03:02:40.197661 3514 leader_election.cc:304] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9533c58a138b4d0e9137093637002f32; no voters:
I20251209 03:02:40.197751 3602 raft_consensus.cc:2804] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:40.197798 3602 raft_consensus.cc:697] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 1 LEADER]: Becoming Leader. State: Replica: 9533c58a138b4d0e9137093637002f32, State: Running, Role: LEADER
I20251209 03:02:40.197842 3602 consensus_queue.cc:237] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.198385 3296 catalog_manager.cc:5654] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9533c58a138b4d0e9137093637002f32 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "9533c58a138b4d0e9137093637002f32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:40.201738 3603 raft_consensus.cc:493] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:40.201786 3603 raft_consensus.cc:515] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.201911 3603 leader_election.cc:290] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.205153 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad233cf84ae24f6499c25e0fa6da860b" candidate_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32" is_pre_election: true
I20251209 03:02:40.205205 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad233cf84ae24f6499c25e0fa6da860b" candidate_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80" is_pre_election: true
I20251209 03:02:40.205214 3550 raft_consensus.cc:2468] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fe5c66eaaa4b2b90253fd6a0d713d1 in term 0.
I20251209 03:02:40.205286 3399 raft_consensus.cc:2468] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fe5c66eaaa4b2b90253fd6a0d713d1 in term 0.
I20251209 03:02:40.205387 3440 leader_election.cc:304] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9533c58a138b4d0e9137093637002f32; no voters:
I20251209 03:02:40.205483 3603 raft_consensus.cc:2804] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:40.205518 3603 raft_consensus.cc:493] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:40.205530 3603 raft_consensus.cc:3060] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.206064 3603 raft_consensus.cc:515] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.206166 3603 leader_election.cc:290] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [CANDIDATE]: Term 1 election: Requested vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.206286 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad233cf84ae24f6499c25e0fa6da860b" candidate_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32"
I20251209 03:02:40.206297 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad233cf84ae24f6499c25e0fa6da860b" candidate_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80"
I20251209 03:02:40.206347 3550 raft_consensus.cc:3060] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.206352 3399 raft_consensus.cc:3060] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.206801 3399 raft_consensus.cc:2468] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fe5c66eaaa4b2b90253fd6a0d713d1 in term 1.
I20251209 03:02:40.206838 3550 raft_consensus.cc:2468] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fe5c66eaaa4b2b90253fd6a0d713d1 in term 1.
I20251209 03:02:40.206962 3439 leader_election.cc:304] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.207058 3603 raft_consensus.cc:2804] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:40.207096 3603 raft_consensus.cc:697] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 LEADER]: Becoming Leader. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Running, Role: LEADER
I20251209 03:02:40.207129 3603 consensus_queue.cc:237] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.207641 3296 catalog_manager.cc:5654] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: UNKNOWN } } }
W20251209 03:02:40.226830 3426 tablet.cc:2378] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251209 03:02:40.414170 3603 raft_consensus.cc:493] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:40.414270 3603 raft_consensus.cc:515] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.414455 3603 leader_election.cc:290] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.414630 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af124c3de2584f59938f5c799d8b9a63" candidate_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80" is_pre_election: true
I20251209 03:02:40.414634 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af124c3de2584f59938f5c799d8b9a63" candidate_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32" is_pre_election: true
I20251209 03:02:40.414716 3399 raft_consensus.cc:2468] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fe5c66eaaa4b2b90253fd6a0d713d1 in term 0.
I20251209 03:02:40.414749 3550 raft_consensus.cc:2468] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32fe5c66eaaa4b2b90253fd6a0d713d1 in term 0.
I20251209 03:02:40.414856 3439 leader_election.cc:304] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.414948 3603 raft_consensus.cc:2804] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:40.414992 3603 raft_consensus.cc:493] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:40.415012 3603 raft_consensus.cc:3060] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.415581 3603 raft_consensus.cc:515] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.415694 3603 leader_election.cc:290] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [CANDIDATE]: Term 1 election: Requested vote from peers 9ea557de30c24f65b427f0908c290a80 (127.2.170.193:46489), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.415804 3399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af124c3de2584f59938f5c799d8b9a63" candidate_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea557de30c24f65b427f0908c290a80"
I20251209 03:02:40.415850 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af124c3de2584f59938f5c799d8b9a63" candidate_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32"
I20251209 03:02:40.415858 3399 raft_consensus.cc:3060] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.415925 3550 raft_consensus.cc:3060] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.416359 3399 raft_consensus.cc:2468] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fe5c66eaaa4b2b90253fd6a0d713d1 in term 1.
I20251209 03:02:40.416416 3550 raft_consensus.cc:2468] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32fe5c66eaaa4b2b90253fd6a0d713d1 in term 1.
I20251209 03:02:40.416522 3439 leader_election.cc:304] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32fe5c66eaaa4b2b90253fd6a0d713d1, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.416620 3603 raft_consensus.cc:2804] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:40.416774 3603 raft_consensus.cc:697] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 LEADER]: Becoming Leader. State: Replica: 32fe5c66eaaa4b2b90253fd6a0d713d1, State: Running, Role: LEADER
I20251209 03:02:40.416833 3603 consensus_queue.cc:237] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.417384 3296 catalog_manager.cc:5654] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: UNKNOWN } } }
I20251209 03:02:40.436870 3613 raft_consensus.cc:493] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:40.436960 3613 raft_consensus.cc:515] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.437101 3613 leader_election.cc:290] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.437281 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "351addfed22a4cfca1193384c0e2dcb0" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32" is_pre_election: true
I20251209 03:02:40.437287 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "351addfed22a4cfca1193384c0e2dcb0" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" is_pre_election: true
I20251209 03:02:40.437350 3550 raft_consensus.cc:2468] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 0.
I20251209 03:02:40.437388 3475 raft_consensus.cc:2468] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 0.
I20251209 03:02:40.437495 3364 leader_election.cc:304] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9533c58a138b4d0e9137093637002f32, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.437582 3613 raft_consensus.cc:2804] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:40.437623 3613 raft_consensus.cc:493] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:40.437638 3613 raft_consensus.cc:3060] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.438179 3613 raft_consensus.cc:515] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.438284 3613 leader_election.cc:290] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 election: Requested vote from peers 32fe5c66eaaa4b2b90253fd6a0d713d1 (127.2.170.194:43185), 9533c58a138b4d0e9137093637002f32 (127.2.170.195:40363)
I20251209 03:02:40.438395 3475 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "351addfed22a4cfca1193384c0e2dcb0" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1"
I20251209 03:02:40.438426 3550 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "351addfed22a4cfca1193384c0e2dcb0" candidate_uuid: "9ea557de30c24f65b427f0908c290a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9533c58a138b4d0e9137093637002f32"
I20251209 03:02:40.438460 3475 raft_consensus.cc:3060] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.438462 3550 raft_consensus.cc:3060] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:40.439029 3475 raft_consensus.cc:2468] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 1.
I20251209 03:02:40.439036 3550 raft_consensus.cc:2468] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ea557de30c24f65b427f0908c290a80 in term 1.
I20251209 03:02:40.439185 3364 leader_election.cc:304] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9533c58a138b4d0e9137093637002f32, 9ea557de30c24f65b427f0908c290a80; no voters:
I20251209 03:02:40.439276 3613 raft_consensus.cc:2804] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:40.439330 3613 raft_consensus.cc:697] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 1 LEADER]: Becoming Leader. State: Replica: 9ea557de30c24f65b427f0908c290a80, State: Running, Role: LEADER
I20251209 03:02:40.439375 3613 consensus_queue.cc:237] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } }
I20251209 03:02:40.439893 3296 catalog_manager.cc:5654] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ea557de30c24f65b427f0908c290a80 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "9ea557de30c24f65b427f0908c290a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: UNKNOWN } } }
I20251209 03:02:40.474098 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:40.475468 3644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:40.475603 3646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:40.475566 3643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:40.475667 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:40.475817 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:40.475844 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:40.475869 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249360475870 us; error 0 us; skew 500 ppm
I20251209 03:02:40.476511 2731 webserver.cc:492] Webserver started at http://127.2.170.196:46053/ using document root <none> and password file <none>
I20251209 03:02:40.476584 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:40.476631 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:40.476678 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:40.476912 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-3-root/instance:
uuid: "bb0aacb15a7d4099b2372b2ac691218f"
format_stamp: "Formatted at 2025-12-09 03:02:40 on dist-test-slave-4cb0"
I20251209 03:02:40.477716 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:40.478159 3651 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.478264 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.478305 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-3-root
uuid: "bb0aacb15a7d4099b2372b2ac691218f"
format_stamp: "Formatted at 2025-12-09 03:02:40 on dist-test-slave-4cb0"
I20251209 03:02:40.478344 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingTurnOffAndOn.1765249356537226-2731-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:40.485560 3602 consensus_queue.cc:1048] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.486799 3602 consensus_queue.cc:1048] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.487293 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:40.487511 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:40.487646 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:40.487833 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:40.487869 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.487898 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:40.487919 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:40.491015 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.196:42895
I20251209 03:02:40.491068 3719 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.196:42895 every 8 connection(s)
I20251209 03:02:40.493775 3720 heartbeater.cc:344] Connected to a master server at 127.2.170.254:43279
I20251209 03:02:40.493824 3720 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:40.493913 3720 heartbeater.cc:507] Master 127.2.170.254:43279 requested a full tablet report, sending...
I20251209 03:02:40.494089 3295 ts_manager.cc:194] Registered new tserver with Master: bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196:42895)
I20251209 03:02:40.494397 3295 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44852
I20251209 03:02:40.502604 3613 consensus_queue.cc:1048] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.503705 3617 consensus_queue.cc:1048] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.599170 3617 consensus_queue.cc:1048] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.600585 3613 consensus_queue.cc:1048] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.630916 3633 consensus_queue.cc:1048] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.632108 3602 consensus_queue.cc:1048] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.643970 3603 consensus_queue.cc:1048] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.645043 3603 consensus_queue.cc:1048] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.675343 3633 consensus_queue.cc:1048] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.676687 3732 consensus_queue.cc:1048] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.831638 3603 consensus_queue.cc:1048] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.833017 3606 consensus_queue.cc:1048] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.947912 3350 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:02:40.948135 3350 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:02:40.948171 3350 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:02:40.951431 3613 consensus_queue.cc:1048] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:40.952860 3617 consensus_queue.cc:1048] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:41.494791 3720 heartbeater.cc:499] Master 127.2.170.254:43279 was elected leader, sending a full tablet report...
I20251209 03:02:42.948312 3350 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:02:42.948496 3350 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:02:42.948523 3350 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:02:44.948653 3350 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:02:44.948812 3350 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:02:44.948839 3350 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:02:45.955097 3475 consensus_queue.cc:237] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } }
I20251209 03:02:45.955695 3550 raft_consensus.cc:1275] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Refusing update from remote peer 32fe5c66eaaa4b2b90253fd6a0d713d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:45.955823 3399 raft_consensus.cc:1275] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Refusing update from remote peer 32fe5c66eaaa4b2b90253fd6a0d713d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:45.956017 3745 consensus_queue.cc:1048] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:02:45.956215 3744 consensus_queue.cc:1048] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:02:45.956727 3745 raft_consensus.cc:2955] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.957751 3399 raft_consensus.cc:2955] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.957971 3296 catalog_manager.cc:5654] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 reported cstate change: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251209 03:02:45.958911 3439 consensus_peers.cc:597] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 -> Peer bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196:42895): Couldn't send request to peer bb0aacb15a7d4099b2372b2ac691218f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad233cf84ae24f6499c25e0fa6da860b. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:02:45.959059 3550 raft_consensus.cc:2955] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.961477 3550 consensus_queue.cc:237] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } attrs { replace: true } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } }
I20251209 03:02:45.962147 3475 raft_consensus.cc:1275] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Refusing update from remote peer 9533c58a138b4d0e9137093637002f32: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:45.962152 3399 raft_consensus.cc:1275] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Refusing update from remote peer 9533c58a138b4d0e9137093637002f32: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:45.962352 3757 consensus_queue.cc:1048] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:02:45.962512 3756 consensus_queue.cc:1048] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:02:45.962980 3757 raft_consensus.cc:2955] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } attrs { replace: true } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.963977 3296 catalog_manager.cc:5654] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 reported cstate change: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "9533c58a138b4d0e9137093637002f32" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:02:45.964092 3475 raft_consensus.cc:2955] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } attrs { replace: true } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.964542 3399 raft_consensus.cc:2955] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } attrs { replace: true } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
W20251209 03:02:45.964871 3514 consensus_peers.cc:597] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 -> Peer bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196:42895): Couldn't send request to peer bb0aacb15a7d4099b2372b2ac691218f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 488d620d01094c2888abd1dd3bce3313. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:02:45.967432 3399 consensus_queue.cc:237] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } attrs { replace: true } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } }
I20251209 03:02:45.967891 3550 raft_consensus.cc:1275] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Refusing update from remote peer 9ea557de30c24f65b427f0908c290a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:45.968030 3475 raft_consensus.cc:1275] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Refusing update from remote peer 9ea557de30c24f65b427f0908c290a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:45.968047 3617 consensus_queue.cc:1048] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:02:45.968160 3617 consensus_queue.cc:1048] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:02:45.968725 3763 raft_consensus.cc:2955] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } attrs { replace: true } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.968828 3550 raft_consensus.cc:2955] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } attrs { replace: true } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.969740 3295 catalog_manager.cc:5654] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 reported cstate change: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "9ea557de30c24f65b427f0908c290a80" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } attrs { replace: true } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.969830 3475 raft_consensus.cc:2955] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } attrs { replace: true } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.969941 3550 consensus_queue.cc:237] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } }
I20251209 03:02:45.970451 3474 raft_consensus.cc:1275] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Refusing update from remote peer 9533c58a138b4d0e9137093637002f32: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251209 03:02:45.970469 3514 consensus_peers.cc:597] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 -> Peer bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196:42895): Couldn't send request to peer bb0aacb15a7d4099b2372b2ac691218f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf09779c2d284f9baa2d1f51a18641bd. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:02:45.970603 3756 consensus_queue.cc:1048] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:02:45.970703 3399 raft_consensus.cc:1275] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Refusing update from remote peer 9533c58a138b4d0e9137093637002f32: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251209 03:02:45.971012 3363 consensus_peers.cc:597] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 -> Peer bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196:42895): Couldn't send request to peer bb0aacb15a7d4099b2372b2ac691218f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e41c5d567cf643fbbe0c2089d57d0dfa. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:02:45.971056 3757 consensus_queue.cc:1048] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:02:45.971429 3756 raft_consensus.cc:2955] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.971537 3474 raft_consensus.cc:2955] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.971735 3399 raft_consensus.cc:2955] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } } }
I20251209 03:02:45.972478 3294 catalog_manager.cc:5654] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 reported cstate change: config changed from index -1 to 2, NON_VOTER bb0aacb15a7d4099b2372b2ac691218f (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "9533c58a138b4d0e9137093637002f32" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9ea557de30c24f65b427f0908c290a80" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 46489 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32fe5c66eaaa4b2b90253fd6a0d713d1" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9533c58a138b4d0e9137093637002f32" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb0aacb15a7d4099b2372b2ac691218f" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42895 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:02:46.003666 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
W20251209 03:02:46.004135 3281 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20251209 03:02:46.004645 3345 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251209 03:02:46.004726 3345 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251209 03:02:46.006814 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:46.006906 2731 tablet_replica.cc:333] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80: stopping tablet replica
I20251209 03:02:46.006963 2731 raft_consensus.cc:2243] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.007025 2731 raft_consensus.cc:2272] T 488d620d01094c2888abd1dd3bce3313 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.007267 2731 tablet_replica.cc:333] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80: stopping tablet replica
I20251209 03:02:46.007323 2731 raft_consensus.cc:2243] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.007402 2731 raft_consensus.cc:2272] T e41c5d567cf643fbbe0c2089d57d0dfa P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.007611 2731 tablet_replica.cc:333] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80: stopping tablet replica
I20251209 03:02:46.007659 2731 raft_consensus.cc:2243] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.007688 2731 raft_consensus.cc:2272] T af124c3de2584f59938f5c799d8b9a63 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.007807 2731 tablet_replica.cc:333] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80: stopping tablet replica
I20251209 03:02:46.007848 2731 raft_consensus.cc:2243] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.007895 2731 raft_consensus.cc:2272] T da77480623e44fd9ad9c75acdfa7939c P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.008052 2731 tablet_replica.cc:333] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80: stopping tablet replica
I20251209 03:02:46.008095 2731 raft_consensus.cc:2243] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.008126 2731 raft_consensus.cc:2272] T a3bf844993c44f5e90c8e2c2388f89b7 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.008227 2731 tablet_replica.cc:333] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80: stopping tablet replica
I20251209 03:02:46.008256 2731 raft_consensus.cc:2243] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.008291 2731 raft_consensus.cc:2272] T 351addfed22a4cfca1193384c0e2dcb0 P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.008356 2731 tablet_replica.cc:333] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80: stopping tablet replica
I20251209 03:02:46.008374 2731 raft_consensus.cc:2243] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.008392 2731 raft_consensus.cc:2272] T ad233cf84ae24f6499c25e0fa6da860b P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.008567 2731 tablet_replica.cc:333] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80: stopping tablet replica
I20251209 03:02:46.008612 2731 raft_consensus.cc:2243] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.008651 2731 raft_consensus.cc:2272] T bf09779c2d284f9baa2d1f51a18641bd P 9ea557de30c24f65b427f0908c290a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.010829 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:02:46.011837 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
W20251209 03:02:46.013056 3345 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251209 03:02:46.013113 3345 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251209 03:02:46.015563 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:46.015707 2731 tablet_replica.cc:333] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1: stopping tablet replica
I20251209 03:02:46.015759 2731 raft_consensus.cc:2243] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.015797 2731 raft_consensus.cc:2272] T 351addfed22a4cfca1193384c0e2dcb0 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.015915 2731 tablet_replica.cc:333] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1: stopping tablet replica
I20251209 03:02:46.015960 2731 raft_consensus.cc:2243] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.015987 2731 raft_consensus.cc:2272] T da77480623e44fd9ad9c75acdfa7939c P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.016091 2731 tablet_replica.cc:333] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1: stopping tablet replica
I20251209 03:02:46.016132 2731 raft_consensus.cc:2243] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.016180 2731 raft_consensus.cc:2272] T af124c3de2584f59938f5c799d8b9a63 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.016283 2731 tablet_replica.cc:333] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1: stopping tablet replica
I20251209 03:02:46.016321 2731 raft_consensus.cc:2243] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.016362 2731 raft_consensus.cc:2272] T e41c5d567cf643fbbe0c2089d57d0dfa P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.016521 2731 tablet_replica.cc:333] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1: stopping tablet replica
I20251209 03:02:46.016574 2731 raft_consensus.cc:2243] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.016637 2731 raft_consensus.cc:2272] T 488d620d01094c2888abd1dd3bce3313 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.016804 2731 tablet_replica.cc:333] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1: stopping tablet replica
I20251209 03:02:46.016839 2731 raft_consensus.cc:2243] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.016863 2731 raft_consensus.cc:2272] T a3bf844993c44f5e90c8e2c2388f89b7 P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.016959 2731 tablet_replica.cc:333] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1: stopping tablet replica
I20251209 03:02:46.016997 2731 raft_consensus.cc:2243] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.017035 2731 raft_consensus.cc:2272] T bf09779c2d284f9baa2d1f51a18641bd P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.017186 2731 tablet_replica.cc:333] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1: stopping tablet replica
I20251209 03:02:46.017226 2731 raft_consensus.cc:2243] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.017288 2731 raft_consensus.cc:2272] T ad233cf84ae24f6499c25e0fa6da860b P 32fe5c66eaaa4b2b90253fd6a0d713d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.029469 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:02:46.030814 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
W20251209 03:02:46.031319 3345 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251209 03:02:46.031371 3345 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251209 03:02:46.031975 3345 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20251209 03:02:46.032021 3345 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20251209 03:02:46.033779 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:46.033875 2731 tablet_replica.cc:333] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32: stopping tablet replica
I20251209 03:02:46.033933 2731 raft_consensus.cc:2243] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.033993 2731 raft_consensus.cc:2272] T a3bf844993c44f5e90c8e2c2388f89b7 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.034111 2731 tablet_replica.cc:333] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32: stopping tablet replica
I20251209 03:02:46.034140 2731 raft_consensus.cc:2243] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.034165 2731 raft_consensus.cc:2272] T e41c5d567cf643fbbe0c2089d57d0dfa P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.034334 2731 tablet_replica.cc:333] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32: stopping tablet replica
I20251209 03:02:46.034370 2731 raft_consensus.cc:2243] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.034394 2731 raft_consensus.cc:2272] T da77480623e44fd9ad9c75acdfa7939c P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.034469 2731 tablet_replica.cc:333] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32: stopping tablet replica
I20251209 03:02:46.034493 2731 raft_consensus.cc:2243] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.034513 2731 raft_consensus.cc:2272] T af124c3de2584f59938f5c799d8b9a63 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.034582 2731 tablet_replica.cc:333] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32: stopping tablet replica
I20251209 03:02:46.034605 2731 raft_consensus.cc:2243] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.034648 2731 raft_consensus.cc:2272] T 488d620d01094c2888abd1dd3bce3313 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.034802 2731 tablet_replica.cc:333] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32: stopping tablet replica
I20251209 03:02:46.034837 2731 raft_consensus.cc:2243] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.034869 2731 raft_consensus.cc:2272] T 351addfed22a4cfca1193384c0e2dcb0 P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.034945 2731 tablet_replica.cc:333] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32: stopping tablet replica
I20251209 03:02:46.034967 2731 raft_consensus.cc:2243] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:46.034991 2731 raft_consensus.cc:2272] T ad233cf84ae24f6499c25e0fa6da860b P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.035157 2731 tablet_replica.cc:333] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32: stopping tablet replica
I20251209 03:02:46.035194 2731 raft_consensus.cc:2243] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.035235 2731 raft_consensus.cc:2272] T bf09779c2d284f9baa2d1f51a18641bd P 9533c58a138b4d0e9137093637002f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.046943 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:02:46.048013 2731 tablet_server.cc:178] TabletServer@127.2.170.196:0 shutting down...
I20251209 03:02:46.050058 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:46.061117 2731 tablet_server.cc:195] TabletServer@127.2.170.196:0 shutdown complete.
I20251209 03:02:46.061801 2731 master.cc:561] Master@127.2.170.254:43279 shutting down...
I20251209 03:02:46.063565 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:46.063635 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:46.063663 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P f187b6dc264346ae9500ed084d70135d: stopping tablet replica
I20251209 03:02:46.075143 2731 master.cc:583] Master@127.2.170.254:43279 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingTurnOffAndOn (7156 ms)
[ RUN ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing
I20251209 03:02:46.081712 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:46273,127.2.170.253:42329,127.2.170.252:33841
I20251209 03:02:46.081851 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:46.083051 3772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:46.083098 3775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:46.083160 3773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.083273 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:46.083384 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:46.083413 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:46.083436 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249366083435 us; error 0 us; skew 500 ppm
I20251209 03:02:46.083921 2731 webserver.cc:492] Webserver started at http://127.2.170.254:45745/ using document root <none> and password file <none>
I20251209 03:02:46.083989 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:46.084029 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:46.084072 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:46.084299 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "2cb6e197fa474ee8af12e69c22c34c7a"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.085018 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:46.085379 3780 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.085494 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.085531 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "2cb6e197fa474ee8af12e69c22c34c7a"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.085572 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:46.090760 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:46.090896 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:46.093050 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:46273
I20251209 03:02:46.093556 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:02:46.097781 3832 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:46273 every 8 connection(s)
I20251209 03:02:46.097903 3833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:46.098543 3833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } has no permanent_uuid. Determining permanent_uuid...
W20251209 03:02:46.098558 3836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:46.098664 3835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.098789 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:46.099337 3838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.099462 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:46.099502 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:46.099519 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249366099519 us; error 0 us; skew 500 ppm
I20251209 03:02:46.100001 2731 webserver.cc:492] Webserver started at http://127.2.170.253:35803/ using document root <none> and password file <none>
I20251209 03:02:46.100080 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:46.100117 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:46.100155 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:46.100375 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root/instance:
uuid: "cd582934493c4c7d85c0b54a8e8f9894"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.101436 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:46.102008 3845 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.102133 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.102159 3833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:46.102186 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root
uuid: "cd582934493c4c7d85c0b54a8e8f9894"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.102231 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20251209 03:02:46.102646 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.170.253:42329: Network error: Client connection negotiation failed: client connection to 127.2.170.253:42329: connect: Connection refused (error 111)
I20251209 03:02:46.136627 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:46.136842 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:46.139498 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.253:42329
I20251209 03:02:46.139534 3898 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.253:42329 every 8 connection(s)
I20251209 03:02:46.139823 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:02:46.139873 3899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:46.140750 3899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } has no permanent_uuid. Determining permanent_uuid...
W20251209 03:02:46.140895 3904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:46.140941 3902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:46.141182 3901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.141444 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:46.141520 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:46.141546 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:46.141562 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249366141562 us; error 0 us; skew 500 ppm
I20251209 03:02:46.142150 2731 webserver.cc:492] Webserver started at http://127.2.170.252:42373/ using document root <none> and password file <none>
I20251209 03:02:46.142228 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:46.142270 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:46.142313 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:46.142527 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root/instance:
uuid: "db7059e3041243c191917b037373d189"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.143383 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:46.143858 3899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:46.143913 3910 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.144044 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.144089 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root
uuid: "db7059e3041243c191917b037373d189"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.144134 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:46.148367 3899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:46.148782 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } attempt: 1
W20251209 03:02:46.149041 3899 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.170.252:33841: Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
I20251209 03:02:46.150447 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:46.150646 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:46.151250 3833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } has no permanent_uuid. Determining permanent_uuid...
W20251209 03:02:46.151726 3833 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.170.252:33841: Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
I20251209 03:02:46.153235 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.252:33841
I20251209 03:02:46.153280 3964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.252:33841 every 8 connection(s)
I20251209 03:02:46.153410 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:02:46.153488 3965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:46.154088 3965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:46.156260 3965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:46.158352 3965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } has no permanent_uuid. Determining permanent_uuid...
I20251209 03:02:46.161074 3965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189: Bootstrap starting.
I20251209 03:02:46.161315 3965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:46.161761 3965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189: No bootstrap required, opened a new log
I20251209 03:02:46.161875 3965 raft_consensus.cc:359] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.161917 3965 raft_consensus.cc:385] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:46.161929 3965 raft_consensus.cc:740] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db7059e3041243c191917b037373d189, State: Initialized, Role: FOLLOWER
I20251209 03:02:46.161963 3965 consensus_queue.cc:260] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.162114 3965 sys_catalog.cc:565] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:02:46.162117 3970 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.162190 3970 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: This master's current role is: FOLLOWER
W20251209 03:02:46.163175 3981 catalog_manager.cc:1568] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251209 03:02:46.163215 3981 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251209 03:02:46.198868 3833 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } attempt: 1
I20251209 03:02:46.202056 3833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: Bootstrap starting.
I20251209 03:02:46.202360 3833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:46.202884 3833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: No bootstrap required, opened a new log
I20251209 03:02:46.203009 3833 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.203066 3833 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:46.203078 3833 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb6e197fa474ee8af12e69c22c34c7a, State: Initialized, Role: FOLLOWER
I20251209 03:02:46.203123 3833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.203301 3833 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:02:46.203318 3983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.203408 3983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:46.204336 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251209 03:02:46.204438 3994 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251209 03:02:46.204470 3994 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251209 03:02:46.209175 3899 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } attempt: 1
I20251209 03:02:46.211983 3899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894: Bootstrap starting.
I20251209 03:02:46.212255 3899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:46.212703 3899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894: No bootstrap required, opened a new log
I20251209 03:02:46.212821 3899 raft_consensus.cc:359] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.212865 3899 raft_consensus.cc:385] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:46.212878 3899 raft_consensus.cc:740] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd582934493c4c7d85c0b54a8e8f9894, State: Initialized, Role: FOLLOWER
I20251209 03:02:46.212911 3899 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.213052 3899 sys_catalog.cc:565] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:02:46.213055 3996 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.213146 3996 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:46.214085 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20251209 03:02:46.214186 4007 catalog_manager.cc:1568] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251209 03:02:46.214226 4007 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251209 03:02:46.214321 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:46.215489 4011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:46.215518 4009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.215638 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:46.215708 4008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.215898 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:46.215929 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:46.215942 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249366215942 us; error 0 us; skew 500 ppm
I20251209 03:02:46.216511 2731 webserver.cc:492] Webserver started at http://127.2.170.193:38685/ using document root <none> and password file <none>
I20251209 03:02:46.216575 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:46.216614 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:46.216658 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:46.216882 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "2385ce1af86c4e8badbf1ac224e0e1b4"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.217579 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:46.217988 4016 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.218106 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.218142 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "2385ce1af86c4e8badbf1ac224e0e1b4"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.218178 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:46.219556 3970 raft_consensus.cc:493] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:46.219601 3970 raft_consensus.cc:515] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.219727 3970 leader_election.cc:290] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cb6e197fa474ee8af12e69c22c34c7a (127.2.170.254:46273), cd582934493c4c7d85c0b54a8e8f9894 (127.2.170.253:42329)
I20251209 03:02:46.219862 3873 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db7059e3041243c191917b037373d189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd582934493c4c7d85c0b54a8e8f9894" is_pre_election: true
I20251209 03:02:46.219867 3807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db7059e3041243c191917b037373d189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" is_pre_election: true
I20251209 03:02:46.219942 3873 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db7059e3041243c191917b037373d189 in term 0.
I20251209 03:02:46.219947 3807 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db7059e3041243c191917b037373d189 in term 0.
I20251209 03:02:46.220084 3914 leader_election.cc:304] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd582934493c4c7d85c0b54a8e8f9894, db7059e3041243c191917b037373d189; no voters:
I20251209 03:02:46.220196 3970 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:46.220223 3970 raft_consensus.cc:493] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:46.220237 3970 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:46.220757 3970 raft_consensus.cc:515] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.220872 3970 leader_election.cc:290] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [CANDIDATE]: Term 1 election: Requested vote from peers 2cb6e197fa474ee8af12e69c22c34c7a (127.2.170.254:46273), cd582934493c4c7d85c0b54a8e8f9894 (127.2.170.253:42329)
I20251209 03:02:46.220949 3807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db7059e3041243c191917b037373d189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb6e197fa474ee8af12e69c22c34c7a"
I20251209 03:02:46.221006 3807 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:46.221026 3873 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db7059e3041243c191917b037373d189" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd582934493c4c7d85c0b54a8e8f9894"
I20251209 03:02:46.221068 3873 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:46.221594 3873 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db7059e3041243c191917b037373d189 in term 1.
I20251209 03:02:46.221791 3807 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db7059e3041243c191917b037373d189 in term 1.
I20251209 03:02:46.221776 3914 leader_election.cc:304] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd582934493c4c7d85c0b54a8e8f9894, db7059e3041243c191917b037373d189; no voters:
I20251209 03:02:46.221923 3970 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:46.222028 3970 raft_consensus.cc:697] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 1 LEADER]: Becoming Leader. State: Replica: db7059e3041243c191917b037373d189, State: Running, Role: LEADER
I20251209 03:02:46.222079 3970 consensus_queue.cc:237] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:46.222311 4021 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db7059e3041243c191917b037373d189. Latest consensus state: current_term: 1 leader_uuid: "db7059e3041243c191917b037373d189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.222375 4021 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:46.222502 4023 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:02:46.222625 4023 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:02:46.223140 3873 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 1 FOLLOWER]: Refusing update from remote peer db7059e3041243c191917b037373d189: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:46.223142 3807 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 1 FOLLOWER]: Refusing update from remote peer db7059e3041243c191917b037373d189: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:02:46.223315 4021 consensus_queue.cc:1048] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:46.223408 3970 consensus_queue.cc:1048] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:46.223824 3996 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db7059e3041243c191917b037373d189. Latest consensus state: current_term: 1 leader_uuid: "db7059e3041243c191917b037373d189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.223886 3983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader db7059e3041243c191917b037373d189. Latest consensus state: current_term: 1 leader_uuid: "db7059e3041243c191917b037373d189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.223964 3983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:46.223891 3996 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:46.224310 4021 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db7059e3041243c191917b037373d189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.224375 4021 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:46.224391 3983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db7059e3041243c191917b037373d189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.224421 4021 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db7059e3041243c191917b037373d189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.224463 4021 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:46.224437 3983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:46.224771 3996 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db7059e3041243c191917b037373d189" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:46.224839 3996 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:46.225104 4023 catalog_manager.cc:1357] Generated new cluster ID: 11f66a94e0f24ac59df8325570457b23
I20251209 03:02:46.225149 4023 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:02:46.228767 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:46.228999 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:46.229141 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:46.229323 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:46.229357 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.229379 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:46.229403 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.232874 4023 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:02:46.233119 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:33109
I20251209 03:02:46.233397 4023 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:02:46.233429 4091 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:33109 every 8 connection(s)
I20251209 03:02:46.233570 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:46.235193 4101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.236898 4092 heartbeater.cc:344] Connected to a master server at 127.2.170.252:33841
W20251209 03:02:46.236912 4099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.236953 4092 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.237121 4092 heartbeater.cc:507] Master 127.2.170.252:33841 requested a full tablet report, sending...
I20251209 03:02:46.237346 3923 ts_manager.cc:194] Registered new tserver with Master: 2385ce1af86c4e8badbf1ac224e0e1b4 (127.2.170.193:33109)
I20251209 03:02:46.237627 4093 heartbeater.cc:344] Connected to a master server at 127.2.170.254:46273
I20251209 03:02:46.237713 4093 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.237803 4093 heartbeater.cc:507] Master 127.2.170.254:46273 requested a full tablet report, sending...
W20251209 03:02:46.236932 4098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.238044 3797 ts_manager.cc:194] Registered new tserver with Master: 2385ce1af86c4e8badbf1ac224e0e1b4 (127.2.170.193:33109)
I20251209 03:02:46.238615 4094 heartbeater.cc:344] Connected to a master server at 127.2.170.253:42329
I20251209 03:02:46.238672 4094 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.238761 4094 heartbeater.cc:507] Master 127.2.170.253:42329 requested a full tablet report, sending...
I20251209 03:02:46.238937 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:46.238993 3863 ts_manager.cc:194] Registered new tserver with Master: 2385ce1af86c4e8badbf1ac224e0e1b4 (127.2.170.193:33109)
I20251209 03:02:46.239053 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:46.239074 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:46.239099 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249366239099 us; error 0 us; skew 500 ppm
I20251209 03:02:46.239625 2731 webserver.cc:492] Webserver started at http://127.2.170.194:39237/ using document root <none> and password file <none>
I20251209 03:02:46.239691 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:46.239734 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:46.239778 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:46.240005 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "b54492fda9a64cc99db6bb942bb4b4a7"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.240743 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:46.241128 4107 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.241255 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.241294 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "b54492fda9a64cc99db6bb942bb4b4a7"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.241334 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:46.245246 4023 catalog_manager.cc:6027] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189: Generated new TSK 0
I20251209 03:02:46.245314 4023 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:02:46.259625 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:46.259835 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:46.259976 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:46.260144 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:46.260170 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.260196 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:46.260221 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.263409 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:41281
I20251209 03:02:46.263477 4170 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:41281 every 8 connection(s)
I20251209 03:02:46.263875 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:46.265188 4181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.265723 2731 server_base.cc:1047] running on GCE node
W20251209 03:02:46.265910 4178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:46.266135 4179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:46.266954 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:46.267001 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:46.267021 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249366267022 us; error 0 us; skew 500 ppm
I20251209 03:02:46.267333 4173 heartbeater.cc:344] Connected to a master server at 127.2.170.253:42329
I20251209 03:02:46.267395 4173 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.267504 4173 heartbeater.cc:507] Master 127.2.170.253:42329 requested a full tablet report, sending...
I20251209 03:02:46.267570 4172 heartbeater.cc:344] Connected to a master server at 127.2.170.254:46273
I20251209 03:02:46.267616 4172 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.267652 2731 webserver.cc:492] Webserver started at http://127.2.170.195:46041/ using document root <none> and password file <none>
I20251209 03:02:46.267709 4172 heartbeater.cc:507] Master 127.2.170.254:46273 requested a full tablet report, sending...
I20251209 03:02:46.267719 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:46.267757 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:46.267815 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:46.267760 3863 ts_manager.cc:194] Registered new tserver with Master: b54492fda9a64cc99db6bb942bb4b4a7 (127.2.170.194:41281)
I20251209 03:02:46.267892 4171 heartbeater.cc:344] Connected to a master server at 127.2.170.252:33841
I20251209 03:02:46.267951 4171 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.268033 4171 heartbeater.cc:507] Master 127.2.170.252:33841 requested a full tablet report, sending...
I20251209 03:02:46.268039 3797 ts_manager.cc:194] Registered new tserver with Master: b54492fda9a64cc99db6bb942bb4b4a7 (127.2.170.194:41281)
I20251209 03:02:46.268097 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "f4bbccc71cfe47c3bda0ad117448a305"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.268210 3923 ts_manager.cc:194] Registered new tserver with Master: b54492fda9a64cc99db6bb942bb4b4a7 (127.2.170.194:41281)
I20251209 03:02:46.268730 3923 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49952
I20251209 03:02:46.268966 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:46.269436 4186 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.269598 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.269644 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "f4bbccc71cfe47c3bda0ad117448a305"
format_stamp: "Formatted at 2025-12-09 03:02:46 on dist-test-slave-4cb0"
I20251209 03:02:46.269704 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NextLeaderResumesAutoRebalancing.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:46.284459 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:46.284698 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:46.284848 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:46.285027 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:46.285054 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.285084 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:46.285107 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:46.288193 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:34445
I20251209 03:02:46.288300 4249 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:34445 every 8 connection(s)
I20251209 03:02:46.291736 4251 heartbeater.cc:344] Connected to a master server at 127.2.170.254:46273
I20251209 03:02:46.291798 4251 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.291831 4252 heartbeater.cc:344] Connected to a master server at 127.2.170.253:42329
I20251209 03:02:46.291888 4252 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.291906 4251 heartbeater.cc:507] Master 127.2.170.254:46273 requested a full tablet report, sending...
I20251209 03:02:46.291987 4252 heartbeater.cc:507] Master 127.2.170.253:42329 requested a full tablet report, sending...
I20251209 03:02:46.292023 4250 heartbeater.cc:344] Connected to a master server at 127.2.170.252:33841
I20251209 03:02:46.292070 4250 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:46.292150 4250 heartbeater.cc:507] Master 127.2.170.252:33841 requested a full tablet report, sending...
I20251209 03:02:46.292201 3797 ts_manager.cc:194] Registered new tserver with Master: f4bbccc71cfe47c3bda0ad117448a305 (127.2.170.195:34445)
I20251209 03:02:46.292346 3923 ts_manager.cc:194] Registered new tserver with Master: f4bbccc71cfe47c3bda0ad117448a305 (127.2.170.195:34445)
I20251209 03:02:46.292362 3863 ts_manager.cc:194] Registered new tserver with Master: f4bbccc71cfe47c3bda0ad117448a305 (127.2.170.195:34445)
I20251209 03:02:46.292690 3923 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49968
I20251209 03:02:46.293138 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004681729s
I20251209 03:02:47.204684 3994 catalog_manager.cc:1269] Loaded cluster ID: 11f66a94e0f24ac59df8325570457b23
I20251209 03:02:47.204761 3994 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: loading cluster ID for follower catalog manager: success
I20251209 03:02:47.205453 3994 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: acquiring CA information for follower catalog manager: success
I20251209 03:02:47.205605 3994 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251209 03:02:47.214392 4007 catalog_manager.cc:1269] Loaded cluster ID: 11f66a94e0f24ac59df8325570457b23
I20251209 03:02:47.214443 4007 catalog_manager.cc:1562] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894: loading cluster ID for follower catalog manager: success
I20251209 03:02:47.215134 4007 catalog_manager.cc:1584] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894: acquiring CA information for follower catalog manager: success
I20251209 03:02:47.215242 4007 catalog_manager.cc:1612] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251209 03:02:47.238341 3923 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49950
I20251209 03:02:47.269174 4171 heartbeater.cc:499] Master 127.2.170.252:33841 was elected leader, sending a full tablet report...
I20251209 03:02:47.293085 4250 heartbeater.cc:499] Master 127.2.170.252:33841 was elected leader, sending a full tablet report...
I20251209 03:02:48.163113 3980 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:02:48.238889 4092 heartbeater.cc:499] Master 127.2.170.252:33841 was elected leader, sending a full tablet report...
I20251209 03:02:48.343182 2731 test_util.cc:276] Using random seed: 629817996
I20251209 03:02:48.348556 3923 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49978:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251209 03:02:48.354038 4135 tablet_service.cc:1505] Processing CreateTablet for tablet 1750af99d306473ab10e41bbe223ca13 (DEFAULT_TABLE table=test-workload [id=81261037a8534d1b9d393be2ff61abf0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251209 03:02:48.354182 4135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1750af99d306473ab10e41bbe223ca13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:48.354583 4056 tablet_service.cc:1505] Processing CreateTablet for tablet 81f68e43062643279eff66cb02e1cda3 (DEFAULT_TABLE table=test-workload [id=81261037a8534d1b9d393be2ff61abf0]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251209 03:02:48.354691 4056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f68e43062643279eff66cb02e1cda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:48.354782 4214 tablet_service.cc:1505] Processing CreateTablet for tablet 7bfddd80fa494e26914d03bbab6022d7 (DEFAULT_TABLE table=test-workload [id=81261037a8534d1b9d393be2ff61abf0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251209 03:02:48.354887 4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bfddd80fa494e26914d03bbab6022d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:48.355630 4286 tablet_bootstrap.cc:492] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7: Bootstrap starting.
I20251209 03:02:48.355924 4287 tablet_bootstrap.cc:492] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4: Bootstrap starting.
I20251209 03:02:48.356200 4286 tablet_bootstrap.cc:654] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:48.356356 4287 tablet_bootstrap.cc:654] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:48.356483 4288 tablet_bootstrap.cc:492] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305: Bootstrap starting.
I20251209 03:02:48.356676 4286 tablet_bootstrap.cc:492] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7: No bootstrap required, opened a new log
I20251209 03:02:48.356720 4286 ts_tablet_manager.cc:1403] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:48.356859 4287 tablet_bootstrap.cc:492] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4: No bootstrap required, opened a new log
I20251209 03:02:48.356874 4286 raft_consensus.cc:359] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54492fda9a64cc99db6bb942bb4b4a7" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 41281 } }
I20251209 03:02:48.356900 4287 ts_tablet_manager.cc:1403] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:48.356935 4286 raft_consensus.cc:385] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:48.356956 4286 raft_consensus.cc:740] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54492fda9a64cc99db6bb942bb4b4a7, State: Initialized, Role: FOLLOWER
I20251209 03:02:48.357007 4286 consensus_queue.cc:260] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54492fda9a64cc99db6bb942bb4b4a7" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 41281 } }
I20251209 03:02:48.357045 4286 raft_consensus.cc:399] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:48.357035 4287 raft_consensus.cc:359] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2385ce1af86c4e8badbf1ac224e0e1b4" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33109 } }
I20251209 03:02:48.357084 4286 raft_consensus.cc:493] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:48.357101 4287 raft_consensus.cc:385] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:48.357103 4286 raft_consensus.cc:3060] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:48.357134 4287 raft_consensus.cc:740] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2385ce1af86c4e8badbf1ac224e0e1b4, State: Initialized, Role: FOLLOWER
I20251209 03:02:48.357200 4287 consensus_queue.cc:260] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2385ce1af86c4e8badbf1ac224e0e1b4" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33109 } }
I20251209 03:02:48.357240 4287 raft_consensus.cc:399] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:48.357264 4287 raft_consensus.cc:493] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:48.357286 4287 raft_consensus.cc:3060] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:48.357762 4286 raft_consensus.cc:515] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54492fda9a64cc99db6bb942bb4b4a7" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 41281 } }
I20251209 03:02:48.357841 4286 leader_election.cc:304] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b54492fda9a64cc99db6bb942bb4b4a7; no voters:
I20251209 03:02:48.357853 4287 raft_consensus.cc:515] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2385ce1af86c4e8badbf1ac224e0e1b4" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33109 } }
I20251209 03:02:48.357892 4287 leader_election.cc:304] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2385ce1af86c4e8badbf1ac224e0e1b4; no voters:
I20251209 03:02:48.357957 4286 leader_election.cc:290] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:48.357972 4287 leader_election.cc:290] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:48.357996 4291 raft_consensus.cc:2804] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:48.358075 4288 tablet_bootstrap.cc:654] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:48.358078 4287 ts_tablet_manager.cc:1434] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:48.358117 4291 raft_consensus.cc:697] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 1 LEADER]: Becoming Leader. State: Replica: b54492fda9a64cc99db6bb942bb4b4a7, State: Running, Role: LEADER
I20251209 03:02:48.358182 4291 consensus_queue.cc:237] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54492fda9a64cc99db6bb942bb4b4a7" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 41281 } }
I20251209 03:02:48.358078 4286 ts_tablet_manager.cc:1434] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251209 03:02:48.358569 3923 catalog_manager.cc:5654] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b54492fda9a64cc99db6bb942bb4b4a7 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "b54492fda9a64cc99db6bb942bb4b4a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54492fda9a64cc99db6bb942bb4b4a7" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 41281 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:48.358278 4294 raft_consensus.cc:2804] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:48.358774 4294 raft_consensus.cc:697] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 1 LEADER]: Becoming Leader. State: Replica: 2385ce1af86c4e8badbf1ac224e0e1b4, State: Running, Role: LEADER
I20251209 03:02:48.358881 4294 consensus_queue.cc:237] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2385ce1af86c4e8badbf1ac224e0e1b4" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33109 } }
I20251209 03:02:48.360198 3923 catalog_manager.cc:5654] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2385ce1af86c4e8badbf1ac224e0e1b4 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "2385ce1af86c4e8badbf1ac224e0e1b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2385ce1af86c4e8badbf1ac224e0e1b4" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33109 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:48.360503 4288 tablet_bootstrap.cc:492] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305: No bootstrap required, opened a new log
I20251209 03:02:48.360548 4288 ts_tablet_manager.cc:1403] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251209 03:02:48.360688 4288 raft_consensus.cc:359] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bbccc71cfe47c3bda0ad117448a305" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34445 } }
I20251209 03:02:48.360734 4288 raft_consensus.cc:385] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:48.360752 4288 raft_consensus.cc:740] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bbccc71cfe47c3bda0ad117448a305, State: Initialized, Role: FOLLOWER
I20251209 03:02:48.360790 4288 consensus_queue.cc:260] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bbccc71cfe47c3bda0ad117448a305" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34445 } }
I20251209 03:02:48.360821 4288 raft_consensus.cc:399] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:48.360837 4288 raft_consensus.cc:493] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:48.360855 4288 raft_consensus.cc:3060] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:48.361263 4288 raft_consensus.cc:515] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bbccc71cfe47c3bda0ad117448a305" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34445 } }
I20251209 03:02:48.361301 4288 leader_election.cc:304] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4bbccc71cfe47c3bda0ad117448a305; no voters:
I20251209 03:02:48.361397 4288 leader_election.cc:290] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:48.361438 4301 raft_consensus.cc:2804] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:48.361485 4288 ts_tablet_manager.cc:1434] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:48.361534 4301 raft_consensus.cc:697] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 1 LEADER]: Becoming Leader. State: Replica: f4bbccc71cfe47c3bda0ad117448a305, State: Running, Role: LEADER
I20251209 03:02:48.361588 4301 consensus_queue.cc:237] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bbccc71cfe47c3bda0ad117448a305" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34445 } }
I20251209 03:02:48.362177 3925 catalog_manager.cc:5654] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4bbccc71cfe47c3bda0ad117448a305 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "f4bbccc71cfe47c3bda0ad117448a305" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bbccc71cfe47c3bda0ad117448a305" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34445 } health_report { overall_health: HEALTHY } } }
I20251209 03:02:48.366892 2731 master.cc:561] Master@127.2.170.252:33841 shutting down...
I20251209 03:02:48.368681 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:48.368743 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:48.368774 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P db7059e3041243c191917b037373d189: stopping tablet replica
I20251209 03:02:48.380762 2731 master.cc:583] Master@127.2.170.252:33841 shutdown complete.
W20251209 03:02:49.360574 4171 heartbeater.cc:646] Failed to heartbeat to 127.2.170.252:33841 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
I20251209 03:02:49.864395 4309 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader db7059e3041243c191917b037373d189)
I20251209 03:02:49.864509 4309 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:49.864631 4310 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader db7059e3041243c191917b037373d189)
I20251209 03:02:49.864764 4310 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:49.864885 3807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd582934493c4c7d85c0b54a8e8f9894" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" is_pre_election: true
I20251209 03:02:49.864930 4310 leader_election.cc:290] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd582934493c4c7d85c0b54a8e8f9894 (127.2.170.253:42329), db7059e3041243c191917b037373d189 (127.2.170.252:33841)
I20251209 03:02:49.864957 3807 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd582934493c4c7d85c0b54a8e8f9894 in term 1.
I20251209 03:02:49.865072 3873 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "cd582934493c4c7d85c0b54a8e8f9894" is_pre_election: true
I20251209 03:02:49.865131 3873 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cb6e197fa474ee8af12e69c22c34c7a in term 1.
I20251209 03:02:49.865262 4309 leader_election.cc:290] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2cb6e197fa474ee8af12e69c22c34c7a (127.2.170.254:46273), db7059e3041243c191917b037373d189 (127.2.170.252:33841)
I20251209 03:02:49.865293 3850 leader_election.cc:304] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb6e197fa474ee8af12e69c22c34c7a, cd582934493c4c7d85c0b54a8e8f9894; no voters:
I20251209 03:02:49.865298 3783 leader_election.cc:304] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb6e197fa474ee8af12e69c22c34c7a, cd582934493c4c7d85c0b54a8e8f9894; no voters:
I20251209 03:02:49.865417 4310 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:02:49.865470 4310 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 1 FOLLOWER]: Starting leader election (detected failure of leader db7059e3041243c191917b037373d189)
I20251209 03:02:49.865494 4310 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 1 FOLLOWER]: Advancing to term 2
W20251209 03:02:49.865799 3782 leader_election.cc:336] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer db7059e3041243c191917b037373d189 (127.2.170.252:33841): Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
W20251209 03:02:49.865964 3848 leader_election.cc:336] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer db7059e3041243c191917b037373d189 (127.2.170.252:33841): Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
I20251209 03:02:49.865407 4309 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:02:49.866042 4309 raft_consensus.cc:493] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 1 FOLLOWER]: Starting leader election (detected failure of leader db7059e3041243c191917b037373d189)
I20251209 03:02:49.866066 4309 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:02:49.866338 4310 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:49.866456 4310 leader_election.cc:290] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 2 election: Requested vote from peers cd582934493c4c7d85c0b54a8e8f9894 (127.2.170.253:42329), db7059e3041243c191917b037373d189 (127.2.170.252:33841)
I20251209 03:02:49.866612 3873 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "cd582934493c4c7d85c0b54a8e8f9894"
I20251209 03:02:49.866685 4309 raft_consensus.cc:515] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:49.866787 3873 raft_consensus.cc:2393] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2cb6e197fa474ee8af12e69c22c34c7a in current term 2: Already voted for candidate cd582934493c4c7d85c0b54a8e8f9894 in this term.
I20251209 03:02:49.866807 4309 leader_election.cc:290] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [CANDIDATE]: Term 2 election: Requested vote from peers 2cb6e197fa474ee8af12e69c22c34c7a (127.2.170.254:46273), db7059e3041243c191917b037373d189 (127.2.170.252:33841)
W20251209 03:02:49.867023 3782 leader_election.cc:336] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer db7059e3041243c191917b037373d189 (127.2.170.252:33841): Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
I20251209 03:02:49.867081 3782 leader_election.cc:304] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2cb6e197fa474ee8af12e69c22c34c7a; no voters: cd582934493c4c7d85c0b54a8e8f9894, db7059e3041243c191917b037373d189
I20251209 03:02:49.867161 4310 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251209 03:02:49.867229 3807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd582934493c4c7d85c0b54a8e8f9894" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "2cb6e197fa474ee8af12e69c22c34c7a"
I20251209 03:02:49.867291 3807 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cd582934493c4c7d85c0b54a8e8f9894 in current term 2: Already voted for candidate 2cb6e197fa474ee8af12e69c22c34c7a in this term.
W20251209 03:02:49.867528 3848 leader_election.cc:336] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer db7059e3041243c191917b037373d189 (127.2.170.252:33841): Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
I20251209 03:02:49.867569 3848 leader_election.cc:304] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd582934493c4c7d85c0b54a8e8f9894; no voters: 2cb6e197fa474ee8af12e69c22c34c7a, db7059e3041243c191917b037373d189
I20251209 03:02:49.867633 4309 raft_consensus.cc:2749] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251209 03:02:51.675209 4322 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:51.675309 4322 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:51.675458 4322 leader_election.cc:290] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd582934493c4c7d85c0b54a8e8f9894 (127.2.170.253:42329), db7059e3041243c191917b037373d189 (127.2.170.252:33841)
I20251209 03:02:51.675696 3873 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "cd582934493c4c7d85c0b54a8e8f9894" is_pre_election: true
I20251209 03:02:51.675770 3873 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cb6e197fa474ee8af12e69c22c34c7a in term 2.
W20251209 03:02:51.675858 3782 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20251209 03:02:51.675918 3783 leader_election.cc:304] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb6e197fa474ee8af12e69c22c34c7a, cd582934493c4c7d85c0b54a8e8f9894; no voters:
I20251209 03:02:51.676019 4322 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251209 03:02:51.676059 4322 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:51.676077 4322 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 2 FOLLOWER]: Advancing to term 3
W20251209 03:02:51.676226 3782 leader_election.cc:336] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer db7059e3041243c191917b037373d189 (127.2.170.252:33841): Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
I20251209 03:02:51.678126 4322 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:51.678246 4322 leader_election.cc:290] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 3 election: Requested vote from peers cd582934493c4c7d85c0b54a8e8f9894 (127.2.170.253:42329), db7059e3041243c191917b037373d189 (127.2.170.252:33841)
I20251209 03:02:51.678378 3873 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "cd582934493c4c7d85c0b54a8e8f9894"
I20251209 03:02:51.678455 3873 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 2 FOLLOWER]: Advancing to term 3
W20251209 03:02:51.678696 3782 leader_election.cc:336] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer db7059e3041243c191917b037373d189 (127.2.170.252:33841): Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111)
I20251209 03:02:51.679044 3873 raft_consensus.cc:2468] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cb6e197fa474ee8af12e69c22c34c7a in term 3.
I20251209 03:02:51.679198 3783 leader_election.cc:304] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2cb6e197fa474ee8af12e69c22c34c7a, cd582934493c4c7d85c0b54a8e8f9894; no voters: db7059e3041243c191917b037373d189
I20251209 03:02:51.679277 4322 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:02:51.679409 4322 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 3 LEADER]: Becoming Leader. State: Replica: 2cb6e197fa474ee8af12e69c22c34c7a, State: Running, Role: LEADER
I20251209 03:02:51.679473 4322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } }
I20251209 03:02:51.679736 4325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cb6e197fa474ee8af12e69c22c34c7a. Latest consensus state: current_term: 3 leader_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:51.679822 4325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:51.679962 4327 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:02:51.680181 4327 catalog_manager.cc:679] Loaded metadata for table test-workload [id=81261037a8534d1b9d393be2ff61abf0]
I20251209 03:02:51.680287 4327 tablet_loader.cc:96] loaded metadata for tablet 1750af99d306473ab10e41bbe223ca13 (table test-workload [id=81261037a8534d1b9d393be2ff61abf0])
I20251209 03:02:51.680327 4327 tablet_loader.cc:96] loaded metadata for tablet 7bfddd80fa494e26914d03bbab6022d7 (table test-workload [id=81261037a8534d1b9d393be2ff61abf0])
I20251209 03:02:51.680361 4327 tablet_loader.cc:96] loaded metadata for tablet 81f68e43062643279eff66cb02e1cda3 (table test-workload [id=81261037a8534d1b9d393be2ff61abf0])
I20251209 03:02:51.680392 4327 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:02:51.680449 4327 catalog_manager.cc:1269] Loaded cluster ID: 11f66a94e0f24ac59df8325570457b23
I20251209 03:02:51.680475 4327 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:02:51.680647 4327 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:02:51.680714 4327 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: Loaded TSK: 0
I20251209 03:02:51.680820 4327 catalog_manager.cc:1524] Initializing in-progress tserver states...
W20251209 03:02:52.085175 3782 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a -> Peer db7059e3041243c191917b037373d189 (127.2.170.252:33841): Couldn't send request to peer db7059e3041243c191917b037373d189. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.252:33841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:02:52.204651 3993 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:02:52.204757 3993 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:02:52.303256 3873 raft_consensus.cc:1275] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 3 FOLLOWER]: Refusing update from remote peer 2cb6e197fa474ee8af12e69c22c34c7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20251209 03:02:52.303520 4325 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20251209 03:02:52.303980 4328 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cb6e197fa474ee8af12e69c22c34c7a. Latest consensus state: current_term: 3 leader_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:52.304070 4328 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:52.304477 4325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:52.304569 4325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:52.304715 4328 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cb6e197fa474ee8af12e69c22c34c7a" member_type: VOTER last_known_addr { host: "127.2.170.254" port: 46273 } } peers { permanent_uuid: "cd582934493c4c7d85c0b54a8e8f9894" member_type: VOTER last_known_addr { host: "127.2.170.253" port: 42329 } } peers { permanent_uuid: "db7059e3041243c191917b037373d189" member_type: VOTER last_known_addr { host: "127.2.170.252" port: 33841 } } }
I20251209 03:02:52.304813 4328 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [sys.catalog]: This master's current role is: FOLLOWER
I20251209 03:02:52.712092 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:02:52.714470 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:52.714597 2731 tablet_replica.cc:333] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4: stopping tablet replica
I20251209 03:02:52.714651 2731 raft_consensus.cc:2243] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:52.714700 2731 raft_consensus.cc:2272] T 81f68e43062643279eff66cb02e1cda3 P 2385ce1af86c4e8badbf1ac224e0e1b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:52.726181 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:02:52.727057 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:02:52.729238 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:52.729344 2731 tablet_replica.cc:333] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7: stopping tablet replica
I20251209 03:02:52.729403 2731 raft_consensus.cc:2243] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:52.729439 2731 raft_consensus.cc:2272] T 1750af99d306473ab10e41bbe223ca13 P b54492fda9a64cc99db6bb942bb4b4a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:52.740675 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:02:52.741456 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:02:52.744040 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:02:52.744130 2731 tablet_replica.cc:333] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305: stopping tablet replica
I20251209 03:02:52.744179 2731 raft_consensus.cc:2243] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:02:52.744225 2731 raft_consensus.cc:2272] T 7bfddd80fa494e26914d03bbab6022d7 P f4bbccc71cfe47c3bda0ad117448a305 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:52.755569 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:02:52.756341 2731 master.cc:561] Master@127.2.170.254:46273 shutting down...
I20251209 03:02:52.758416 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 3 LEADER]: Raft consensus shutting down.
I20251209 03:02:52.758522 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:52.758596 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2cb6e197fa474ee8af12e69c22c34c7a: stopping tablet replica
I20251209 03:02:52.770383 2731 master.cc:583] Master@127.2.170.254:46273 shutdown complete.
I20251209 03:02:52.771517 2731 master.cc:561] Master@127.2.170.253:42329 shutting down...
I20251209 03:02:52.773469 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251209 03:02:52.773540 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:02:52.773557 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P cd582934493c4c7d85c0b54a8e8f9894: stopping tablet replica
I20251209 03:02:52.785246 2731 master.cc:583] Master@127.2.170.253:42329 shutdown complete.
[ OK ] AutoRebalancerTest.NextLeaderResumesAutoRebalancing (6708 ms)
[ RUN ] AutoRebalancerTest.MovesScheduledIfAddTserver
I20251209 03:02:52.789987 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:33257
I20251209 03:02:52.790154 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:52.791291 4340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:52.791373 4339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:52.791373 4342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:52.791553 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:52.792225 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:52.792271 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:52.792286 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249372792287 us; error 0 us; skew 500 ppm
I20251209 03:02:52.792804 2731 webserver.cc:492] Webserver started at http://127.2.170.254:34153/ using document root <none> and password file <none>
I20251209 03:02:52.792886 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:52.792923 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:52.792960 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:52.793267 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "5a13de61cb3b4dd59bd68d7c80b2719e"
format_stamp: "Formatted at 2025-12-09 03:02:52 on dist-test-slave-4cb0"
I20251209 03:02:52.794416 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:52.795006 4347 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.795320 2731 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:52.795368 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "5a13de61cb3b4dd59bd68d7c80b2719e"
format_stamp: "Formatted at 2025-12-09 03:02:52 on dist-test-slave-4cb0"
I20251209 03:02:52.795414 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:52.811547 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:52.811748 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:52.814853 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:33257
I20251209 03:02:52.817663 4399 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:33257 every 8 connection(s)
I20251209 03:02:52.817903 4400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:52.819391 4400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e: Bootstrap starting.
I20251209 03:02:52.819736 4400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:52.820345 4400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e: No bootstrap required, opened a new log
I20251209 03:02:52.820484 4400 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a13de61cb3b4dd59bd68d7c80b2719e" member_type: VOTER }
I20251209 03:02:52.820545 4400 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:52.820565 4400 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a13de61cb3b4dd59bd68d7c80b2719e, State: Initialized, Role: FOLLOWER
I20251209 03:02:52.820638 4400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a13de61cb3b4dd59bd68d7c80b2719e" member_type: VOTER }
I20251209 03:02:52.820680 4400 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:02:52.820709 4400 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:02:52.820736 4400 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:52.821373 4400 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a13de61cb3b4dd59bd68d7c80b2719e" member_type: VOTER }
I20251209 03:02:52.821451 4400 leader_election.cc:304] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a13de61cb3b4dd59bd68d7c80b2719e; no voters:
I20251209 03:02:52.821571 4400 leader_election.cc:290] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:02:52.821631 4403 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:52.821741 4400 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:02:52.821784 4403 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 1 LEADER]: Becoming Leader. State: Replica: 5a13de61cb3b4dd59bd68d7c80b2719e, State: Running, Role: LEADER
I20251209 03:02:52.821834 4403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a13de61cb3b4dd59bd68d7c80b2719e" member_type: VOTER }
I20251209 03:02:52.822041 4404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a13de61cb3b4dd59bd68d7c80b2719e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a13de61cb3b4dd59bd68d7c80b2719e" member_type: VOTER } }
I20251209 03:02:52.822057 4405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a13de61cb3b4dd59bd68d7c80b2719e. Latest consensus state: current_term: 1 leader_uuid: "5a13de61cb3b4dd59bd68d7c80b2719e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a13de61cb3b4dd59bd68d7c80b2719e" member_type: VOTER } }
I20251209 03:02:52.822096 4404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:52.822111 4405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [sys.catalog]: This master's current role is: LEADER
I20251209 03:02:52.822256 4407 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:02:52.822398 4407 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:02:52.822948 4407 catalog_manager.cc:1357] Generated new cluster ID: 8018a93b20c44c6883a44df3a6f7237c
I20251209 03:02:52.822997 4407 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:02:52.823102 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:02:52.831398 4407 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:02:52.832003 4407 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:02:52.840767 4407 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e: Generated new TSK 0
I20251209 03:02:52.840862 4407 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:02:52.854861 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:52.856149 4424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:52.856247 4421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:52.856262 4422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:52.856439 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:52.856528 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:52.856551 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:52.856561 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249372856562 us; error 0 us; skew 500 ppm
I20251209 03:02:52.857085 2731 webserver.cc:492] Webserver started at http://127.2.170.193:37213/ using document root <none> and password file <none>
I20251209 03:02:52.857165 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:52.857205 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:52.857240 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:52.857462 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "28016da1468a4d8aa3a650d68b5ef60d"
format_stamp: "Formatted at 2025-12-09 03:02:52 on dist-test-slave-4cb0"
I20251209 03:02:52.858270 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:52.858647 4429 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.858759 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.858803 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "28016da1468a4d8aa3a650d68b5ef60d"
format_stamp: "Formatted at 2025-12-09 03:02:52 on dist-test-slave-4cb0"
I20251209 03:02:52.858841 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:52.862679 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:52.862859 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:52.862974 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:52.863130 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:52.863154 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.863174 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:52.863197 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.865872 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:33729
I20251209 03:02:52.865900 4492 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:33729 every 8 connection(s)
I20251209 03:02:52.866217 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:52.867251 4498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:52.867327 4500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:52.867373 4497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:52.867754 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:52.867897 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:52.867925 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:52.867942 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249372867942 us; error 0 us; skew 500 ppm
I20251209 03:02:52.868492 2731 webserver.cc:492] Webserver started at http://127.2.170.194:34669/ using document root <none> and password file <none>
I20251209 03:02:52.868558 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:52.868603 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:52.868647 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:52.868928 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "f1662736c00d42c5b54fd0a86310c40c"
format_stamp: "Formatted at 2025-12-09 03:02:52 on dist-test-slave-4cb0"
I20251209 03:02:52.869859 4493 heartbeater.cc:344] Connected to a master server at 127.2.170.254:33257
I20251209 03:02:52.869915 4493 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:52.869992 4493 heartbeater.cc:507] Master 127.2.170.254:33257 requested a full tablet report, sending...
I20251209 03:02:52.869975 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:52.870464 4364 ts_manager.cc:194] Registered new tserver with Master: 28016da1468a4d8aa3a650d68b5ef60d (127.2.170.193:33729)
I20251209 03:02:52.870476 4505 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.870587 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.870623 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "f1662736c00d42c5b54fd0a86310c40c"
format_stamp: "Formatted at 2025-12-09 03:02:52 on dist-test-slave-4cb0"
I20251209 03:02:52.870664 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:52.871246 4364 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44986
I20251209 03:02:52.894951 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:52.895188 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:52.895328 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:52.895535 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:52.895562 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.895591 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:52.895615 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.898298 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:42629
I20251209 03:02:52.898361 4568 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:42629 every 8 connection(s)
I20251209 03:02:52.898672 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:02:52.899849 4573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:52.899880 4572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:02:52.900025 4575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:02:52.900156 2731 server_base.cc:1047] running on GCE node
I20251209 03:02:52.900291 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:02:52.900326 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:02:52.900344 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249372900344 us; error 0 us; skew 500 ppm
I20251209 03:02:52.900988 2731 webserver.cc:492] Webserver started at http://127.2.170.195:46635/ using document root <none> and password file <none>
I20251209 03:02:52.901069 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:02:52.901110 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:02:52.901160 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:02:52.901489 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "ae5727a52a074876af6e701597e27e87"
format_stamp: "Formatted at 2025-12-09 03:02:52 on dist-test-slave-4cb0"
I20251209 03:02:52.902009 4569 heartbeater.cc:344] Connected to a master server at 127.2.170.254:33257
I20251209 03:02:52.902076 4569 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:52.902173 4569 heartbeater.cc:507] Master 127.2.170.254:33257 requested a full tablet report, sending...
I20251209 03:02:52.902367 4364 ts_manager.cc:194] Registered new tserver with Master: f1662736c00d42c5b54fd0a86310c40c (127.2.170.194:42629)
I20251209 03:02:52.902483 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:52.902804 4364 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44992
I20251209 03:02:52.903061 4580 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.903208 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251209 03:02:52.903254 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "ae5727a52a074876af6e701597e27e87"
format_stamp: "Formatted at 2025-12-09 03:02:52 on dist-test-slave-4cb0"
I20251209 03:02:52.903297 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:02:52.922948 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:02:52.923193 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:02:52.923344 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:02:52.923530 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:02:52.923559 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.923588 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:02:52.923610 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:52.926613 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:34135
I20251209 03:02:52.926673 4643 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:34135 every 8 connection(s)
I20251209 03:02:52.929375 4644 heartbeater.cc:344] Connected to a master server at 127.2.170.254:33257
I20251209 03:02:52.929432 4644 heartbeater.cc:461] Registering TS with master...
I20251209 03:02:52.929541 4644 heartbeater.cc:507] Master 127.2.170.254:33257 requested a full tablet report, sending...
I20251209 03:02:52.929774 4364 ts_manager.cc:194] Registered new tserver with Master: ae5727a52a074876af6e701597e27e87 (127.2.170.195:34135)
I20251209 03:02:52.930073 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003291575s
I20251209 03:02:52.930116 4364 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44994
I20251209 03:02:53.871730 4493 heartbeater.cc:499] Master 127.2.170.254:33257 was elected leader, sending a full tablet report...
I20251209 03:02:53.903355 4569 heartbeater.cc:499] Master 127.2.170.254:33257 was elected leader, sending a full tablet report...
I20251209 03:02:53.930624 4644 heartbeater.cc:499] Master 127.2.170.254:33257 was elected leader, sending a full tablet report...
I20251209 03:02:53.954906 2731 test_util.cc:276] Using random seed: 635429720
I20251209 03:02:53.959918 4364 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44998:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:02:53.960096 4364 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:02:53.964747 4457 tablet_service.cc:1505] Processing CreateTablet for tablet aeffa13bc8e24462adbbfdf987cb7ffb (DEFAULT_TABLE table=test-workload [id=d5d06f8c1a484476a06fde02890e6a61]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251209 03:02:53.964896 4457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeffa13bc8e24462adbbfdf987cb7ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:53.964960 4532 tablet_service.cc:1505] Processing CreateTablet for tablet 77ded8de6b5242ea9cb8ade7bf699b19 (DEFAULT_TABLE table=test-workload [id=d5d06f8c1a484476a06fde02890e6a61]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251209 03:02:53.964977 4533 tablet_service.cc:1505] Processing CreateTablet for tablet aeffa13bc8e24462adbbfdf987cb7ffb (DEFAULT_TABLE table=test-workload [id=d5d06f8c1a484476a06fde02890e6a61]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251209 03:02:53.965026 4608 tablet_service.cc:1505] Processing CreateTablet for tablet aeffa13bc8e24462adbbfdf987cb7ffb (DEFAULT_TABLE table=test-workload [id=d5d06f8c1a484476a06fde02890e6a61]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20251209 03:02:53.965034 4532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ded8de6b5242ea9cb8ade7bf699b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:53.965113 4608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeffa13bc8e24462adbbfdf987cb7ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:53.965178 4533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeffa13bc8e24462adbbfdf987cb7ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:53.965751 4607 tablet_service.cc:1505] Processing CreateTablet for tablet 77ded8de6b5242ea9cb8ade7bf699b19 (DEFAULT_TABLE table=test-workload [id=d5d06f8c1a484476a06fde02890e6a61]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251209 03:02:53.965842 4607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ded8de6b5242ea9cb8ade7bf699b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:53.966375 4456 tablet_service.cc:1505] Processing CreateTablet for tablet 77ded8de6b5242ea9cb8ade7bf699b19 (DEFAULT_TABLE table=test-workload [id=d5d06f8c1a484476a06fde02890e6a61]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20251209 03:02:53.966468 4456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ded8de6b5242ea9cb8ade7bf699b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:02:53.966508 4665 tablet_bootstrap.cc:492] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d: Bootstrap starting.
I20251209 03:02:53.967119 4665 tablet_bootstrap.cc:654] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:53.967453 4664 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c: Bootstrap starting.
I20251209 03:02:53.967666 4665 tablet_bootstrap.cc:492] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d: No bootstrap required, opened a new log
I20251209 03:02:53.967705 4665 ts_tablet_manager.cc:1403] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:53.967871 4665 raft_consensus.cc:359] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:53.967931 4665 raft_consensus.cc:385] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:53.967943 4665 raft_consensus.cc:740] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28016da1468a4d8aa3a650d68b5ef60d, State: Initialized, Role: FOLLOWER
I20251209 03:02:53.967975 4665 consensus_queue.cc:260] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:53.968108 4665 ts_tablet_manager.cc:1434] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:02:53.968160 4665 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d: Bootstrap starting.
I20251209 03:02:53.968503 4665 tablet_bootstrap.cc:654] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:53.969048 4665 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d: No bootstrap required, opened a new log
I20251209 03:02:53.969058 4664 tablet_bootstrap.cc:654] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:53.969102 4665 ts_tablet_manager.cc:1403] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:53.969103 4666 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: Bootstrap starting.
I20251209 03:02:53.969285 4665 raft_consensus.cc:359] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.969340 4665 raft_consensus.cc:385] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:53.969358 4665 raft_consensus.cc:740] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28016da1468a4d8aa3a650d68b5ef60d, State: Initialized, Role: FOLLOWER
I20251209 03:02:53.969405 4665 consensus_queue.cc:260] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.969499 4665 ts_tablet_manager.cc:1434] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:53.969587 4666 tablet_bootstrap.cc:654] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:53.970163 4664 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c: No bootstrap required, opened a new log
I20251209 03:02:53.970209 4664 ts_tablet_manager.cc:1403] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251209 03:02:53.970329 4664 raft_consensus.cc:359] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.970371 4664 raft_consensus.cc:385] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:53.970389 4664 raft_consensus.cc:740] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1662736c00d42c5b54fd0a86310c40c, State: Initialized, Role: FOLLOWER
I20251209 03:02:53.970438 4664 consensus_queue.cc:260] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.970579 4664 ts_tablet_manager.cc:1434] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:02:53.970630 4664 tablet_bootstrap.cc:492] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c: Bootstrap starting.
I20251209 03:02:53.970999 4664 tablet_bootstrap.cc:654] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:53.971357 4666 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: No bootstrap required, opened a new log
I20251209 03:02:53.971393 4666 ts_tablet_manager.cc:1403] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20251209 03:02:53.971495 4666 raft_consensus.cc:359] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.971520 4664 tablet_bootstrap.cc:492] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c: No bootstrap required, opened a new log
I20251209 03:02:53.971554 4666 raft_consensus.cc:385] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:53.971572 4666 raft_consensus.cc:740] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae5727a52a074876af6e701597e27e87, State: Initialized, Role: FOLLOWER
I20251209 03:02:53.971577 4664 ts_tablet_manager.cc:1403] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:02:53.971611 4666 consensus_queue.cc:260] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.971742 4664 raft_consensus.cc:359] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:53.971743 4666 ts_tablet_manager.cc:1434] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:53.971866 4666 tablet_bootstrap.cc:492] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87: Bootstrap starting.
I20251209 03:02:53.971805 4664 raft_consensus.cc:385] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:53.971916 4664 raft_consensus.cc:740] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1662736c00d42c5b54fd0a86310c40c, State: Initialized, Role: FOLLOWER
I20251209 03:02:53.971971 4664 consensus_queue.cc:260] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:53.972060 4664 ts_tablet_manager.cc:1434] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:53.972254 4666 tablet_bootstrap.cc:654] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87: Neither blocks nor log segments found. Creating new log.
I20251209 03:02:53.972784 4666 tablet_bootstrap.cc:492] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87: No bootstrap required, opened a new log
I20251209 03:02:53.972822 4666 ts_tablet_manager.cc:1403] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:02:53.972945 4666 raft_consensus.cc:359] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:53.972991 4666 raft_consensus.cc:385] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:02:53.973002 4666 raft_consensus.cc:740] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae5727a52a074876af6e701597e27e87, State: Initialized, Role: FOLLOWER
I20251209 03:02:53.973033 4666 consensus_queue.cc:260] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:53.973098 4666 ts_tablet_manager.cc:1434] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:02:53.985773 4672 raft_consensus.cc:493] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:53.985855 4672 raft_consensus.cc:515] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.986019 4672 leader_election.cc:290] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28016da1468a4d8aa3a650d68b5ef60d (127.2.170.193:33729), ae5727a52a074876af6e701597e27e87 (127.2.170.195:34135)
I20251209 03:02:53.988507 4467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77ded8de6b5242ea9cb8ade7bf699b19" candidate_uuid: "f1662736c00d42c5b54fd0a86310c40c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28016da1468a4d8aa3a650d68b5ef60d" is_pre_election: true
I20251209 03:02:53.988597 4618 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77ded8de6b5242ea9cb8ade7bf699b19" candidate_uuid: "f1662736c00d42c5b54fd0a86310c40c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae5727a52a074876af6e701597e27e87" is_pre_election: true
I20251209 03:02:53.988615 4467 raft_consensus.cc:2468] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1662736c00d42c5b54fd0a86310c40c in term 0.
I20251209 03:02:53.988642 4618 raft_consensus.cc:2468] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1662736c00d42c5b54fd0a86310c40c in term 0.
I20251209 03:02:53.988763 4508 leader_election.cc:304] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28016da1468a4d8aa3a650d68b5ef60d, f1662736c00d42c5b54fd0a86310c40c; no voters:
I20251209 03:02:53.988878 4672 raft_consensus.cc:2804] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:53.988945 4672 raft_consensus.cc:493] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:53.988967 4672 raft_consensus.cc:3060] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:53.989807 4672 raft_consensus.cc:515] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.989941 4672 leader_election.cc:290] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [CANDIDATE]: Term 1 election: Requested vote from peers 28016da1468a4d8aa3a650d68b5ef60d (127.2.170.193:33729), ae5727a52a074876af6e701597e27e87 (127.2.170.195:34135)
I20251209 03:02:53.990068 4467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77ded8de6b5242ea9cb8ade7bf699b19" candidate_uuid: "f1662736c00d42c5b54fd0a86310c40c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28016da1468a4d8aa3a650d68b5ef60d"
I20251209 03:02:53.990068 4618 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77ded8de6b5242ea9cb8ade7bf699b19" candidate_uuid: "f1662736c00d42c5b54fd0a86310c40c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae5727a52a074876af6e701597e27e87"
I20251209 03:02:53.990146 4467 raft_consensus.cc:3060] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:53.990156 4618 raft_consensus.cc:3060] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:53.990821 4618 raft_consensus.cc:2468] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1662736c00d42c5b54fd0a86310c40c in term 1.
I20251209 03:02:53.990821 4467 raft_consensus.cc:2468] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1662736c00d42c5b54fd0a86310c40c in term 1.
I20251209 03:02:53.990986 4508 leader_election.cc:304] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28016da1468a4d8aa3a650d68b5ef60d, f1662736c00d42c5b54fd0a86310c40c; no voters:
I20251209 03:02:53.991070 4672 raft_consensus.cc:2804] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:53.991209 4672 raft_consensus.cc:697] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 1 LEADER]: Becoming Leader. State: Replica: f1662736c00d42c5b54fd0a86310c40c, State: Running, Role: LEADER
I20251209 03:02:53.991258 4672 consensus_queue.cc:237] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } }
I20251209 03:02:53.991830 4363 catalog_manager.cc:5654] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c reported cstate change: term changed from 0 to 1, leader changed from <none> to f1662736c00d42c5b54fd0a86310c40c (127.2.170.194). New cstate: current_term: 1 leader_uuid: "f1662736c00d42c5b54fd0a86310c40c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } health_report { overall_health: UNKNOWN } } }
I20251209 03:02:54.051398 4674 raft_consensus.cc:493] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:02:54.051507 4674 raft_consensus.cc:515] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:54.051703 4674 leader_election.cc:290] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1662736c00d42c5b54fd0a86310c40c (127.2.170.194:42629), 28016da1468a4d8aa3a650d68b5ef60d (127.2.170.193:33729)
I20251209 03:02:54.054389 4467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aeffa13bc8e24462adbbfdf987cb7ffb" candidate_uuid: "ae5727a52a074876af6e701597e27e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28016da1468a4d8aa3a650d68b5ef60d" is_pre_election: true
I20251209 03:02:54.054481 4467 raft_consensus.cc:2468] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae5727a52a074876af6e701597e27e87 in term 0.
I20251209 03:02:54.054440 4543 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aeffa13bc8e24462adbbfdf987cb7ffb" candidate_uuid: "ae5727a52a074876af6e701597e27e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1662736c00d42c5b54fd0a86310c40c" is_pre_election: true
I20251209 03:02:54.054541 4543 raft_consensus.cc:2468] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae5727a52a074876af6e701597e27e87 in term 0.
I20251209 03:02:54.054636 4583 leader_election.cc:304] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28016da1468a4d8aa3a650d68b5ef60d, ae5727a52a074876af6e701597e27e87; no voters:
I20251209 03:02:54.054724 4674 raft_consensus.cc:2804] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:02:54.054772 4674 raft_consensus.cc:493] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:02:54.054786 4674 raft_consensus.cc:3060] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:54.055354 4674 raft_consensus.cc:515] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:54.055450 4674 leader_election.cc:290] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [CANDIDATE]: Term 1 election: Requested vote from peers f1662736c00d42c5b54fd0a86310c40c (127.2.170.194:42629), 28016da1468a4d8aa3a650d68b5ef60d (127.2.170.193:33729)
I20251209 03:02:54.055624 4467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aeffa13bc8e24462adbbfdf987cb7ffb" candidate_uuid: "ae5727a52a074876af6e701597e27e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28016da1468a4d8aa3a650d68b5ef60d"
I20251209 03:02:54.055620 4543 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aeffa13bc8e24462adbbfdf987cb7ffb" candidate_uuid: "ae5727a52a074876af6e701597e27e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1662736c00d42c5b54fd0a86310c40c"
I20251209 03:02:54.055694 4467 raft_consensus.cc:3060] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:54.055709 4543 raft_consensus.cc:3060] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:02:54.056211 4467 raft_consensus.cc:2468] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae5727a52a074876af6e701597e27e87 in term 1.
I20251209 03:02:54.056293 4543 raft_consensus.cc:2468] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae5727a52a074876af6e701597e27e87 in term 1.
I20251209 03:02:54.056389 4583 leader_election.cc:304] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28016da1468a4d8aa3a650d68b5ef60d, ae5727a52a074876af6e701597e27e87; no voters:
I20251209 03:02:54.056476 4674 raft_consensus.cc:2804] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:02:54.056602 4674 raft_consensus.cc:697] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 1 LEADER]: Becoming Leader. State: Replica: ae5727a52a074876af6e701597e27e87, State: Running, Role: LEADER
I20251209 03:02:54.056641 4674 consensus_queue.cc:237] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } }
I20251209 03:02:54.057117 4363 catalog_manager.cc:5654] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae5727a52a074876af6e701597e27e87 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "ae5727a52a074876af6e701597e27e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } health_report { overall_health: UNKNOWN } } }
I20251209 03:02:54.455235 4674 consensus_queue.cc:1048] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:54.456897 4682 consensus_queue.cc:1048] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:54.483856 4672 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:54.485248 4690 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:02:54.823125 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:02:54.823230 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:02:54.823247 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:02:56.823429 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:02:56.823562 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:02:56.823601 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:02:58.823757 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:02:58.823900 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:02:58.823936 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:00.824098 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:00.824263 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:00.824297 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:02.824442 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:02.824580 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:02.824626 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:04.824761 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:04.824894 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:04.824927 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:05.126838 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:05.128180 4697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:05.128293 4698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:05.128304 4700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:05.128463 2731 server_base.cc:1047] running on GCE node
I20251209 03:03:05.128582 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:05.128608 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:05.128620 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249385128621 us; error 0 us; skew 500 ppm
I20251209 03:03:05.129245 2731 webserver.cc:492] Webserver started at http://127.2.170.196:45757/ using document root <none> and password file <none>
I20251209 03:03:05.129325 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:05.129369 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:05.129416 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:05.129711 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-3-root/instance:
uuid: "34e5705ebf124fc6b041b65dcaa56613"
format_stamp: "Formatted at 2025-12-09 03:03:05 on dist-test-slave-4cb0"
I20251209 03:03:05.130528 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:03:05.130961 4705 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:05.131070 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:05.131111 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-3-root
uuid: "34e5705ebf124fc6b041b65dcaa56613"
format_stamp: "Formatted at 2025-12-09 03:03:05 on dist-test-slave-4cb0"
I20251209 03:03:05.131156 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:05.137465 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:05.137651 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:05.137831 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:05.138021 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:05.138053 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:05.138073 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:05.138089 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:05.141628 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.196:42391
I20251209 03:03:05.141709 4768 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.196:42391 every 8 connection(s)
I20251209 03:03:05.144444 4769 heartbeater.cc:344] Connected to a master server at 127.2.170.254:33257
I20251209 03:03:05.144503 4769 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:05.144608 4769 heartbeater.cc:507] Master 127.2.170.254:33257 requested a full tablet report, sending...
I20251209 03:03:05.144829 4364 ts_manager.cc:194] Registered new tserver with Master: 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196:42391)
I20251209 03:03:05.145197 4364 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38838
I20251209 03:03:05.828311 4543 consensus_queue.cc:237] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: true } }
I20251209 03:03:05.828868 4467 raft_consensus.cc:1275] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Refusing update from remote peer f1662736c00d42c5b54fd0a86310c40c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:03:05.828934 4618 raft_consensus.cc:1275] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Refusing update from remote peer f1662736c00d42c5b54fd0a86310c40c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:03:05.829063 4777 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:03:05.829190 4778 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:03:05.829948 4773 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: true } } }
I20251209 03:03:05.830061 4467 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: true } } }
I20251209 03:03:05.830706 4618 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: true } } }
I20251209 03:03:05.831112 4364 catalog_manager.cc:5654] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c reported cstate change: config changed from index -1 to 2, NON_VOTER 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "f1662736c00d42c5b54fd0a86310c40c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251209 03:03:05.832371 4507 consensus_peers.cc:597] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c -> Peer 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196:42391): Couldn't send request to peer 34e5705ebf124fc6b041b65dcaa56613. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77ded8de6b5242ea9cb8ade7bf699b19. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:03:06.145699 4769 heartbeater.cc:499] Master 127.2.170.254:33257 was elected leader, sending a full tablet report...
I20251209 03:03:06.418859 4788 ts_tablet_manager.cc:933] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: Initiating tablet copy from peer f1662736c00d42c5b54fd0a86310c40c (127.2.170.194:42629)
I20251209 03:03:06.419076 4788 tablet_copy_client.cc:323] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42629
I20251209 03:03:06.421654 4553 tablet_copy_service.cc:140] P f1662736c00d42c5b54fd0a86310c40c: Received BeginTabletCopySession request for tablet 77ded8de6b5242ea9cb8ade7bf699b19 from peer 34e5705ebf124fc6b041b65dcaa56613 ({username='slave'} at 127.0.0.1:34926)
I20251209 03:03:06.421769 4553 tablet_copy_service.cc:161] P f1662736c00d42c5b54fd0a86310c40c: Beginning new tablet copy session on tablet 77ded8de6b5242ea9cb8ade7bf699b19 from peer 34e5705ebf124fc6b041b65dcaa56613 at {username='slave'} at 127.0.0.1:34926: session id = 34e5705ebf124fc6b041b65dcaa56613-77ded8de6b5242ea9cb8ade7bf699b19
I20251209 03:03:06.422412 4553 tablet_copy_source_session.cc:215] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:03:06.422876 4788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ded8de6b5242ea9cb8ade7bf699b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:06.423846 4788 tablet_copy_client.cc:806] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: tablet copy: Starting download of 0 data blocks...
I20251209 03:03:06.423975 4788 tablet_copy_client.cc:670] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: tablet copy: Starting download of 1 WAL segments...
I20251209 03:03:06.424654 4788 tablet_copy_client.cc:538] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:03:06.425531 4788 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: Bootstrap starting.
I20251209 03:03:06.426946 4788 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:03:06.427114 4788 tablet_bootstrap.cc:492] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: Bootstrap complete.
I20251209 03:03:06.427199 4788 ts_tablet_manager.cc:1403] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251209 03:03:06.427387 4788 raft_consensus.cc:359] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: true } }
I20251209 03:03:06.427453 4788 raft_consensus.cc:740] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34e5705ebf124fc6b041b65dcaa56613, State: Initialized, Role: LEARNER
I20251209 03:03:06.427481 4788 consensus_queue.cc:260] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: true } }
I20251209 03:03:06.427661 4788 ts_tablet_manager.cc:1434] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:03:06.427845 4553 tablet_copy_service.cc:342] P f1662736c00d42c5b54fd0a86310c40c: Request end of tablet copy session 34e5705ebf124fc6b041b65dcaa56613-77ded8de6b5242ea9cb8ade7bf699b19 received from {username='slave'} at 127.0.0.1:34926
I20251209 03:03:06.427889 4553 tablet_copy_service.cc:434] P f1662736c00d42c5b54fd0a86310c40c: ending tablet copy session 34e5705ebf124fc6b041b65dcaa56613-77ded8de6b5242ea9cb8ade7bf699b19 on tablet 77ded8de6b5242ea9cb8ade7bf699b19 with peer 34e5705ebf124fc6b041b65dcaa56613
I20251209 03:03:06.825052 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:06.825178 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:06.825222 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:06.918089 4743 raft_consensus.cc:1217] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251209 03:03:07.349496 4794 raft_consensus.cc:1064] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c: attempting to promote NON_VOTER 34e5705ebf124fc6b041b65dcaa56613 to VOTER
I20251209 03:03:07.349795 4794 consensus_queue.cc:237] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } }
I20251209 03:03:07.350291 4743 raft_consensus.cc:1275] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 LEARNER]: Refusing update from remote peer f1662736c00d42c5b54fd0a86310c40c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:03:07.350292 4467 raft_consensus.cc:1275] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Refusing update from remote peer f1662736c00d42c5b54fd0a86310c40c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:03:07.350342 4618 raft_consensus.cc:1275] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Refusing update from remote peer f1662736c00d42c5b54fd0a86310c40c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:03:07.350526 4797 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:03:07.350677 4778 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:03:07.350836 4796 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:03:07.351508 4796 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } } }
I20251209 03:03:07.351764 4467 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } } }
I20251209 03:03:07.351605 4743 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } } }
I20251209 03:03:07.352703 4618 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } } }
I20251209 03:03:07.353176 4362 catalog_manager.cc:5654] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c reported cstate change: config changed from index 2 to 3, 34e5705ebf124fc6b041b65dcaa56613 (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f1662736c00d42c5b54fd0a86310c40c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae5727a52a074876af6e701597e27e87" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34135 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:03:07.354761 4542 consensus_queue.cc:237] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } }
I20251209 03:03:07.355084 4467 raft_consensus.cc:1275] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Refusing update from remote peer f1662736c00d42c5b54fd0a86310c40c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:03:07.355178 4743 raft_consensus.cc:1275] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 FOLLOWER]: Refusing update from remote peer f1662736c00d42c5b54fd0a86310c40c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:03:07.355226 4794 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:03:07.355331 4797 consensus_queue.cc:1048] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:03:07.355695 4797 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ae5727a52a074876af6e701597e27e87 (127.2.170.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } } }
I20251209 03:03:07.355934 4467 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ae5727a52a074876af6e701597e27e87 (127.2.170.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } } }
I20251209 03:03:07.356731 4743 raft_consensus.cc:2955] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ae5727a52a074876af6e701597e27e87 (127.2.170.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } } }
I20251209 03:03:07.356770 4350 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 77ded8de6b5242ea9cb8ade7bf699b19 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:03:07.356873 4361 catalog_manager.cc:5654] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c reported cstate change: config changed from index 3 to 4, VOTER ae5727a52a074876af6e701597e27e87 (127.2.170.195) evicted. New cstate: current_term: 1 leader_uuid: "f1662736c00d42c5b54fd0a86310c40c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "28016da1468a4d8aa3a650d68b5ef60d" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 33729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1662736c00d42c5b54fd0a86310c40c" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e5705ebf124fc6b041b65dcaa56613" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 42391 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:03:07.357600 4608 tablet_service.cc:1552] Processing DeleteTablet for tablet 77ded8de6b5242ea9cb8ade7bf699b19 with delete_type TABLET_DATA_TOMBSTONED (TS ae5727a52a074876af6e701597e27e87 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48200
I20251209 03:03:07.357856 4806 tablet_replica.cc:333] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: stopping tablet replica
I20251209 03:03:07.357914 4806 raft_consensus.cc:2243] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:03:07.357964 4806 raft_consensus.cc:2272] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:07.358165 4806 ts_tablet_manager.cc:1916] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:03:07.359184 4806 ts_tablet_manager.cc:1929] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251209 03:03:07.359239 4806 log.cc:1199] T 77ded8de6b5242ea9cb8ade7bf699b19 P ae5727a52a074876af6e701597e27e87: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.MovesScheduledIfAddTserver.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/77ded8de6b5242ea9cb8ade7bf699b19
I20251209 03:03:07.359524 4351 catalog_manager.cc:4985] TS ae5727a52a074876af6e701597e27e87 (127.2.170.195:34135): tablet 77ded8de6b5242ea9cb8ade7bf699b19 (table test-workload [id=d5d06f8c1a484476a06fde02890e6a61]) successfully deleted
I20251209 03:03:08.825361 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:08.825482 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:08.825518 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:10.825742 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:10.825877 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:10.825917 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:12.826066 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:12.826195 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:12.826231 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:14.826440 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:14.826586 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:14.826611 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:16.826786 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:16.826911 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:16.826941 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:18.829245 4419 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:18.829408 4419 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:18.829438 4419 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:19.730533 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:03:19.733250 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:19.733352 2731 tablet_replica.cc:333] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d: stopping tablet replica
I20251209 03:03:19.733419 2731 raft_consensus.cc:2243] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:03:19.733484 2731 raft_consensus.cc:2272] T 77ded8de6b5242ea9cb8ade7bf699b19 P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:19.733711 2731 tablet_replica.cc:333] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d: stopping tablet replica
I20251209 03:03:19.733754 2731 raft_consensus.cc:2243] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:03:19.733789 2731 raft_consensus.cc:2272] T aeffa13bc8e24462adbbfdf987cb7ffb P 28016da1468a4d8aa3a650d68b5ef60d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:19.745417 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:03:19.746284 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:03:19.749133 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:19.749239 2731 tablet_replica.cc:333] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c: stopping tablet replica
I20251209 03:03:19.749293 2731 raft_consensus.cc:2243] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:03:19.749339 2731 raft_consensus.cc:2272] T aeffa13bc8e24462adbbfdf987cb7ffb P f1662736c00d42c5b54fd0a86310c40c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:19.749493 2731 tablet_replica.cc:333] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c: stopping tablet replica
I20251209 03:03:19.749526 2731 raft_consensus.cc:2243] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:03:19.749576 2731 raft_consensus.cc:2272] T 77ded8de6b5242ea9cb8ade7bf699b19 P f1662736c00d42c5b54fd0a86310c40c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:19.761281 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:03:19.762213 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:03:19.764871 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:19.764963 2731 tablet_replica.cc:333] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87: stopping tablet replica
I20251209 03:03:19.765009 2731 raft_consensus.cc:2243] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:03:19.765075 2731 raft_consensus.cc:2272] T aeffa13bc8e24462adbbfdf987cb7ffb P ae5727a52a074876af6e701597e27e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:19.777218 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:03:19.778122 2731 tablet_server.cc:178] TabletServer@127.2.170.196:0 shutting down...
I20251209 03:03:19.780723 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:19.780809 2731 tablet_replica.cc:333] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613: stopping tablet replica
I20251209 03:03:19.780865 2731 raft_consensus.cc:2243] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:03:19.780916 2731 raft_consensus.cc:2272] T 77ded8de6b5242ea9cb8ade7bf699b19 P 34e5705ebf124fc6b041b65dcaa56613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:19.792169 2731 tablet_server.cc:195] TabletServer@127.2.170.196:0 shutdown complete.
I20251209 03:03:19.793018 2731 master.cc:561] Master@127.2.170.254:33257 shutting down...
I20251209 03:03:19.795630 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:03:19.795701 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:19.795723 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5a13de61cb3b4dd59bd68d7c80b2719e: stopping tablet replica
I20251209 03:03:19.807184 2731 master.cc:583] Master@127.2.170.254:33257 shutdown complete.
[ OK ] AutoRebalancerTest.MovesScheduledIfAddTserver (27021 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTservers
I20251209 03:03:19.811712 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:45519
I20251209 03:03:19.811849 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:03:19.813186 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:19.813248 4810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:19.813222 4812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:19.813194 4809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:19.813524 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:19.813557 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:19.813568 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249399813568 us; error 0 us; skew 500 ppm
I20251209 03:03:19.814226 2731 webserver.cc:492] Webserver started at http://127.2.170.254:33375/ using document root <none> and password file <none>
I20251209 03:03:19.814318 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:19.814368 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:19.814400 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:19.814623 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "baa84a18420744178881f932317276b9"
format_stamp: "Formatted at 2025-12-09 03:03:19 on dist-test-slave-4cb0"
I20251209 03:03:19.815400 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:19.815768 4817 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:19.815897 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:19.815944 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "baa84a18420744178881f932317276b9"
format_stamp: "Formatted at 2025-12-09 03:03:19 on dist-test-slave-4cb0"
I20251209 03:03:19.815980 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTservers.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:19.827090 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:19.827258 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:19.829545 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:45519
I20251209 03:03:19.831682 4869 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:45519 every 8 connection(s)
I20251209 03:03:19.831844 4870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:19.832968 4870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9: Bootstrap starting.
I20251209 03:03:19.833245 4870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:19.833724 4870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9: No bootstrap required, opened a new log
I20251209 03:03:19.833851 4870 raft_consensus.cc:359] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa84a18420744178881f932317276b9" member_type: VOTER }
I20251209 03:03:19.833904 4870 raft_consensus.cc:385] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:19.833922 4870 raft_consensus.cc:740] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa84a18420744178881f932317276b9, State: Initialized, Role: FOLLOWER
I20251209 03:03:19.833969 4870 consensus_queue.cc:260] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa84a18420744178881f932317276b9" member_type: VOTER }
I20251209 03:03:19.834007 4870 raft_consensus.cc:399] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:03:19.834026 4870 raft_consensus.cc:493] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:03:19.834052 4870 raft_consensus.cc:3060] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:19.834559 4870 raft_consensus.cc:515] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa84a18420744178881f932317276b9" member_type: VOTER }
I20251209 03:03:19.834617 4870 leader_election.cc:304] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: baa84a18420744178881f932317276b9; no voters:
I20251209 03:03:19.834717 4870 leader_election.cc:290] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:03:19.834762 4873 raft_consensus.cc:2804] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:03:19.834852 4870 sys_catalog.cc:565] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:03:19.834913 4873 raft_consensus.cc:697] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 1 LEADER]: Becoming Leader. State: Replica: baa84a18420744178881f932317276b9, State: Running, Role: LEADER
I20251209 03:03:19.834995 4873 consensus_queue.cc:237] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa84a18420744178881f932317276b9" member_type: VOTER }
I20251209 03:03:19.835202 4874 sys_catalog.cc:455] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "baa84a18420744178881f932317276b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa84a18420744178881f932317276b9" member_type: VOTER } }
I20251209 03:03:19.835215 4875 sys_catalog.cc:455] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader baa84a18420744178881f932317276b9. Latest consensus state: current_term: 1 leader_uuid: "baa84a18420744178881f932317276b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baa84a18420744178881f932317276b9" member_type: VOTER } }
I20251209 03:03:19.835279 4874 sys_catalog.cc:458] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [sys.catalog]: This master's current role is: LEADER
I20251209 03:03:19.835284 4875 sys_catalog.cc:458] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [sys.catalog]: This master's current role is: LEADER
I20251209 03:03:19.835482 4880 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:03:19.835628 4880 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:03:19.836150 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:03:19.836252 4880 catalog_manager.cc:1357] Generated new cluster ID: 647dc9f3185145deb9c9f3d75c919b96
I20251209 03:03:19.836297 4880 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:03:19.851014 4880 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:03:19.851491 4880 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:03:19.854545 4880 catalog_manager.cc:6027] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9: Generated new TSK 0
I20251209 03:03:19.854614 4880 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:03:19.867784 2731 internal_mini_cluster.cc:371] 0 TS(s) registered with all masters after 4.7205e-05s
I20251209 03:03:31.943164 2731 master.cc:561] Master@127.2.170.254:45519 shutting down...
I20251209 03:03:31.945200 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:03:31.945284 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:31.945312 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P baa84a18420744178881f932317276b9: stopping tablet replica
I20251209 03:03:31.956578 2731 master.cc:583] Master@127.2.170.254:45519 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTservers (12147 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfNoTablets
I20251209 03:03:31.958911 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:34487
I20251209 03:03:31.959054 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:31.960278 4893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:31.960392 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:31.960278 4895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:31.960489 4892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:31.960651 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:31.960688 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:31.960700 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249411960700 us; error 0 us; skew 500 ppm
I20251209 03:03:31.961331 2731 webserver.cc:492] Webserver started at http://127.2.170.254:38873/ using document root <none> and password file <none>
I20251209 03:03:31.961401 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:31.961442 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:31.961488 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:31.961756 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "494447a894ba443e88985cb3d8b8f6c6"
format_stamp: "Formatted at 2025-12-09 03:03:31 on dist-test-slave-4cb0"
I20251209 03:03:31.962548 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:03:31.962973 4900 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:31.963073 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:31.963114 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "494447a894ba443e88985cb3d8b8f6c6"
format_stamp: "Formatted at 2025-12-09 03:03:31 on dist-test-slave-4cb0"
I20251209 03:03:31.963157 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:31.969132 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:31.969269 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:31.972142 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:34487
I20251209 03:03:31.972728 4952 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:34487 every 8 connection(s)
I20251209 03:03:31.972819 4953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:31.973727 4953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6: Bootstrap starting.
I20251209 03:03:31.973975 4953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:31.974371 4953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6: No bootstrap required, opened a new log
I20251209 03:03:31.974484 4953 raft_consensus.cc:359] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494447a894ba443e88985cb3d8b8f6c6" member_type: VOTER }
I20251209 03:03:31.974529 4953 raft_consensus.cc:385] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:31.974545 4953 raft_consensus.cc:740] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 494447a894ba443e88985cb3d8b8f6c6, State: Initialized, Role: FOLLOWER
I20251209 03:03:31.974584 4953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494447a894ba443e88985cb3d8b8f6c6" member_type: VOTER }
I20251209 03:03:31.974628 4953 raft_consensus.cc:399] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:03:31.974646 4953 raft_consensus.cc:493] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:03:31.974671 4953 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:31.975102 4953 raft_consensus.cc:515] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494447a894ba443e88985cb3d8b8f6c6" member_type: VOTER }
I20251209 03:03:31.975152 4953 leader_election.cc:304] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 494447a894ba443e88985cb3d8b8f6c6; no voters:
I20251209 03:03:31.975250 4953 leader_election.cc:290] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:03:31.975299 4956 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:03:31.975392 4953 sys_catalog.cc:565] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:03:31.975420 4956 raft_consensus.cc:697] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 1 LEADER]: Becoming Leader. State: Replica: 494447a894ba443e88985cb3d8b8f6c6, State: Running, Role: LEADER
I20251209 03:03:31.975466 4956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494447a894ba443e88985cb3d8b8f6c6" member_type: VOTER }
I20251209 03:03:31.975672 4958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 494447a894ba443e88985cb3d8b8f6c6. Latest consensus state: current_term: 1 leader_uuid: "494447a894ba443e88985cb3d8b8f6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494447a894ba443e88985cb3d8b8f6c6" member_type: VOTER } }
I20251209 03:03:31.975669 4957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "494447a894ba443e88985cb3d8b8f6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494447a894ba443e88985cb3d8b8f6c6" member_type: VOTER } }
I20251209 03:03:31.975786 4957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [sys.catalog]: This master's current role is: LEADER
I20251209 03:03:31.975953 4958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [sys.catalog]: This master's current role is: LEADER
I20251209 03:03:31.975976 4963 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:03:31.976137 4963 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:03:31.976617 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:03:31.976672 4963 catalog_manager.cc:1357] Generated new cluster ID: 35d22625f1ca47bf8ac59f07169a690e
I20251209 03:03:31.976707 4963 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:03:31.980830 4963 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:03:31.981221 4963 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:03:31.986056 4963 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6: Generated new TSK 0
I20251209 03:03:31.986129 4963 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:03:31.992238 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:31.993445 4975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:31.993502 4974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:31.993546 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:31.993728 4977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:31.993835 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:31.993870 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:31.993882 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249411993882 us; error 0 us; skew 500 ppm
I20251209 03:03:31.994334 2731 webserver.cc:492] Webserver started at http://127.2.170.193:42067/ using document root <none> and password file <none>
I20251209 03:03:31.994390 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:31.994419 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:31.994453 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:31.994668 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "2f258b951cb343daa558bbbc63f3288b"
format_stamp: "Formatted at 2025-12-09 03:03:31 on dist-test-slave-4cb0"
I20251209 03:03:31.995489 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:31.995882 4982 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:31.996006 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:31.996058 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "2f258b951cb343daa558bbbc63f3288b"
format_stamp: "Formatted at 2025-12-09 03:03:31 on dist-test-slave-4cb0"
I20251209 03:03:31.996095 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:32.009625 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:32.009822 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:32.009943 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:32.010084 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:32.010106 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.010129 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:32.010145 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.012629 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:33899
I20251209 03:03:32.012681 5045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:33899 every 8 connection(s)
I20251209 03:03:32.012961 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:32.014123 5050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:32.014207 5051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:32.014422 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:32.014577 5053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:32.014719 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:32.014760 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:32.014772 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249412014773 us; error 0 us; skew 500 ppm
I20251209 03:03:32.015467 2731 webserver.cc:492] Webserver started at http://127.2.170.194:46547/ using document root <none> and password file <none>
I20251209 03:03:32.015545 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:32.015590 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:32.015630 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:32.015826 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "039146d17d4e4307af94a43586b692fe"
format_stamp: "Formatted at 2025-12-09 03:03:32 on dist-test-slave-4cb0"
I20251209 03:03:32.016583 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:32.016966 5058 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.017066 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.017110 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "039146d17d4e4307af94a43586b692fe"
format_stamp: "Formatted at 2025-12-09 03:03:32 on dist-test-slave-4cb0"
I20251209 03:03:32.017167 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:32.019136 5046 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34487
I20251209 03:03:32.019196 5046 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:32.019292 5046 heartbeater.cc:507] Master 127.2.170.254:34487 requested a full tablet report, sending...
I20251209 03:03:32.019479 4917 ts_manager.cc:194] Registered new tserver with Master: 2f258b951cb343daa558bbbc63f3288b (127.2.170.193:33899)
I20251209 03:03:32.020025 4917 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36114
I20251209 03:03:32.054554 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:32.054785 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:32.054944 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:32.055138 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:32.055166 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.055194 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:32.055212 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.057869 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:37321
I20251209 03:03:32.057962 5121 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:37321 every 8 connection(s)
I20251209 03:03:32.058261 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:32.059389 5125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:32.059501 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:32.059468 5128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:32.059641 5126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:32.059860 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:32.059902 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:32.059927 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249412059927 us; error 0 us; skew 500 ppm
I20251209 03:03:32.060523 2731 webserver.cc:492] Webserver started at http://127.2.170.195:33397/ using document root <none> and password file <none>
I20251209 03:03:32.060598 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:32.060633 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:32.060669 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:32.060902 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "d17c9c1f1a3d4f3d986a8a57db8824d8"
format_stamp: "Formatted at 2025-12-09 03:03:32 on dist-test-slave-4cb0"
I20251209 03:03:32.061203 5122 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34487
I20251209 03:03:32.061254 5122 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:32.061335 5122 heartbeater.cc:507] Master 127.2.170.254:34487 requested a full tablet report, sending...
I20251209 03:03:32.061546 4917 ts_manager.cc:194] Registered new tserver with Master: 039146d17d4e4307af94a43586b692fe (127.2.170.194:37321)
I20251209 03:03:32.061820 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:32.061904 4917 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36130
I20251209 03:03:32.062310 5133 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.062417 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.062463 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "d17c9c1f1a3d4f3d986a8a57db8824d8"
format_stamp: "Formatted at 2025-12-09 03:03:32 on dist-test-slave-4cb0"
I20251209 03:03:32.062505 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfNoTablets.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:32.097726 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:32.097990 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:32.098152 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:32.098335 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:32.098361 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.098389 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:32.098413 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:32.101446 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:40425
I20251209 03:03:32.101545 5196 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:40425 every 8 connection(s)
I20251209 03:03:32.104153 5197 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34487
I20251209 03:03:32.104208 5197 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:32.104297 5197 heartbeater.cc:507] Master 127.2.170.254:34487 requested a full tablet report, sending...
I20251209 03:03:32.104496 4917 ts_manager.cc:194] Registered new tserver with Master: d17c9c1f1a3d4f3d986a8a57db8824d8 (127.2.170.195:40425)
I20251209 03:03:32.104806 4917 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36144
I20251209 03:03:32.104911 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003322617s
I20251209 03:03:33.020478 5046 heartbeater.cc:499] Master 127.2.170.254:34487 was elected leader, sending a full tablet report...
I20251209 03:03:33.062381 5122 heartbeater.cc:499] Master 127.2.170.254:34487 was elected leader, sending a full tablet report...
I20251209 03:03:33.105262 5197 heartbeater.cc:499] Master 127.2.170.254:34487 was elected leader, sending a full tablet report...
I20251209 03:03:33.976696 4971 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:35.976868 4971 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:37.977044 4971 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:39.977264 4971 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:41.977485 4971 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:43.977706 4971 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:44.180203 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:03:44.182709 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:44.194104 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:03:44.194878 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:03:44.196810 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:44.208019 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:03:44.208747 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:03:44.210841 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:44.222112 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:03:44.222862 2731 master.cc:561] Master@127.2.170.254:34487 shutting down...
I20251209 03:03:44.225142 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:03:44.225215 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:44.225242 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 494447a894ba443e88985cb3d8b8f6c6: stopping tablet replica
I20251209 03:03:44.236697 2731 master.cc:583] Master@127.2.170.254:34487 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfNoTablets (12280 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne
I20251209 03:03:44.239961 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:43379
I20251209 03:03:44.240105 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:03:44.241430 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:44.241529 5201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:44.241544 5204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:44.241622 5202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:44.241796 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:44.241832 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:44.241842 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249424241842 us; error 0 us; skew 500 ppm
I20251209 03:03:44.242348 2731 webserver.cc:492] Webserver started at http://127.2.170.254:42523/ using document root <none> and password file <none>
I20251209 03:03:44.242424 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:44.242463 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:44.242506 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:44.242738 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "f571cca2e4744e12a5425c2aa6e8c59e"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.243543 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:44.243978 5209 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.244103 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.244149 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "f571cca2e4744e12a5425c2aa6e8c59e"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.244190 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:44.251988 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:44.252158 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:44.254546 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:43379
I20251209 03:03:44.255805 5261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:43379 every 8 connection(s)
I20251209 03:03:44.255916 5262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:44.256953 5262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e: Bootstrap starting.
I20251209 03:03:44.257239 5262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:44.257812 5262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e: No bootstrap required, opened a new log
I20251209 03:03:44.257946 5262 raft_consensus.cc:359] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571cca2e4744e12a5425c2aa6e8c59e" member_type: VOTER }
I20251209 03:03:44.258002 5262 raft_consensus.cc:385] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:44.258018 5262 raft_consensus.cc:740] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f571cca2e4744e12a5425c2aa6e8c59e, State: Initialized, Role: FOLLOWER
I20251209 03:03:44.258073 5262 consensus_queue.cc:260] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571cca2e4744e12a5425c2aa6e8c59e" member_type: VOTER }
I20251209 03:03:44.258114 5262 raft_consensus.cc:399] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:03:44.258140 5262 raft_consensus.cc:493] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:03:44.258169 5262 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:44.258649 5262 raft_consensus.cc:515] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571cca2e4744e12a5425c2aa6e8c59e" member_type: VOTER }
I20251209 03:03:44.258716 5262 leader_election.cc:304] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f571cca2e4744e12a5425c2aa6e8c59e; no voters:
I20251209 03:03:44.258823 5262 leader_election.cc:290] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:03:44.258886 5265 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:03:44.258975 5262 sys_catalog.cc:565] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:03:44.259016 5265 raft_consensus.cc:697] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 1 LEADER]: Becoming Leader. State: Replica: f571cca2e4744e12a5425c2aa6e8c59e, State: Running, Role: LEADER
I20251209 03:03:44.259094 5265 consensus_queue.cc:237] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571cca2e4744e12a5425c2aa6e8c59e" member_type: VOTER }
I20251209 03:03:44.259289 5266 sys_catalog.cc:455] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f571cca2e4744e12a5425c2aa6e8c59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571cca2e4744e12a5425c2aa6e8c59e" member_type: VOTER } }
I20251209 03:03:44.259296 5267 sys_catalog.cc:455] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f571cca2e4744e12a5425c2aa6e8c59e. Latest consensus state: current_term: 1 leader_uuid: "f571cca2e4744e12a5425c2aa6e8c59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f571cca2e4744e12a5425c2aa6e8c59e" member_type: VOTER } }
I20251209 03:03:44.259375 5267 sys_catalog.cc:458] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [sys.catalog]: This master's current role is: LEADER
I20251209 03:03:44.259366 5266 sys_catalog.cc:458] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [sys.catalog]: This master's current role is: LEADER
I20251209 03:03:44.259639 5271 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:03:44.259794 5271 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:03:44.260340 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:03:44.260406 5271 catalog_manager.cc:1357] Generated new cluster ID: 66437459e2d1473da858ff51378c3f69
I20251209 03:03:44.260438 5271 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:03:44.273960 5271 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:03:44.274354 5271 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:03:44.278158 5271 catalog_manager.cc:6027] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e: Generated new TSK 0
I20251209 03:03:44.278224 5271 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:03:44.292130 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:44.293547 5286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:44.293572 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:44.293617 5283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:44.293560 5284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:44.293982 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:44.294044 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:44.294059 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249424294059 us; error 0 us; skew 500 ppm
I20251209 03:03:44.294608 2731 webserver.cc:492] Webserver started at http://127.2.170.193:43509/ using document root <none> and password file <none>
I20251209 03:03:44.294687 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:44.294765 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:44.294799 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:44.295046 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "7e682ab9ef4343cbaef1052beeb1e23c"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.295846 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:03:44.296278 5291 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.296389 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.296439 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "7e682ab9ef4343cbaef1052beeb1e23c"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.296479 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:44.305334 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:44.305531 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:44.305655 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:44.305868 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:44.305894 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.305913 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:44.305936 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.308463 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:35925
I20251209 03:03:44.308509 5354 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:35925 every 8 connection(s)
I20251209 03:03:44.308794 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:44.309919 5359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:44.309970 5360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:44.310117 5362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:44.310290 2731 server_base.cc:1047] running on GCE node
I20251209 03:03:44.310403 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:44.310425 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:44.310436 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249424310436 us; error 0 us; skew 500 ppm
I20251209 03:03:44.310930 2731 webserver.cc:492] Webserver started at http://127.2.170.194:45201/ using document root <none> and password file <none>
I20251209 03:03:44.310999 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:44.311040 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:44.311084 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:44.311357 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "865a28131ef54f91b27098e9c848d67e"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.312172 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:44.312283 5355 heartbeater.cc:344] Connected to a master server at 127.2.170.254:43379
I20251209 03:03:44.312331 5355 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:44.312420 5355 heartbeater.cc:507] Master 127.2.170.254:43379 requested a full tablet report, sending...
I20251209 03:03:44.312629 5226 ts_manager.cc:194] Registered new tserver with Master: 7e682ab9ef4343cbaef1052beeb1e23c (127.2.170.193:35925)
I20251209 03:03:44.312654 5367 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.312793 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.312840 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "865a28131ef54f91b27098e9c848d67e"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.312885 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:44.313405 5226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52078
I20251209 03:03:44.326349 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:44.326538 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:44.326659 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:44.326839 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:44.326866 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.326887 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:44.326910 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.329963 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:43457
I20251209 03:03:44.330006 5430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:43457 every 8 connection(s)
I20251209 03:03:44.330304 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:44.331496 5434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:44.331614 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:44.331717 5435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:44.331845 5437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:44.331986 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:44.332038 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:44.332051 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249424332051 us; error 0 us; skew 500 ppm
I20251209 03:03:44.332623 2731 webserver.cc:492] Webserver started at http://127.2.170.195:36533/ using document root <none> and password file <none>
I20251209 03:03:44.332703 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:44.332748 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:44.332795 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:44.333093 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "411777afbe37428ba981aa3acb3b909a"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.333405 5431 heartbeater.cc:344] Connected to a master server at 127.2.170.254:43379
I20251209 03:03:44.333470 5431 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:44.333562 5431 heartbeater.cc:507] Master 127.2.170.254:43379 requested a full tablet report, sending...
I20251209 03:03:44.333777 5226 ts_manager.cc:194] Registered new tserver with Master: 865a28131ef54f91b27098e9c848d67e (127.2.170.194:43457)
I20251209 03:03:44.334113 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:03:44.334187 5226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52086
I20251209 03:03:44.334556 5442 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.334673 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.334713 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "411777afbe37428ba981aa3acb3b909a"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.334755 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:44.348362 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:44.348614 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:44.348790 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:44.348984 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:44.349010 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.349040 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:44.349064 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.352156 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:39895
I20251209 03:03:44.352209 5505 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:39895 every 8 connection(s)
I20251209 03:03:44.352654 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:44.353996 5510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:44.354041 5509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:44.354282 5512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:44.354300 2731 server_base.cc:1047] running on GCE node
I20251209 03:03:44.354494 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:44.354528 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:44.354545 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249424354545 us; error 0 us; skew 500 ppm
I20251209 03:03:44.355199 2731 webserver.cc:492] Webserver started at http://127.2.170.196:40427/ using document root <none> and password file <none>
I20251209 03:03:44.355275 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:44.355321 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:44.355366 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:44.355659 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-3-root/instance:
uuid: "c932fb0decca4ccc84aa06706332a26d"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.355783 5506 heartbeater.cc:344] Connected to a master server at 127.2.170.254:43379
I20251209 03:03:44.355845 5506 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:44.355942 5506 heartbeater.cc:507] Master 127.2.170.254:43379 requested a full tablet report, sending...
I20251209 03:03:44.356151 5226 ts_manager.cc:194] Registered new tserver with Master: 411777afbe37428ba981aa3acb3b909a (127.2.170.195:39895)
I20251209 03:03:44.356623 5226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52088
I20251209 03:03:44.356808 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:44.357273 5517 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.357393 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.357436 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-3-root
uuid: "c932fb0decca4ccc84aa06706332a26d"
format_stamp: "Formatted at 2025-12-09 03:03:44 on dist-test-slave-4cb0"
I20251209 03:03:44.357481 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne.1765249356537226-2731-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:44.368587 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:44.368829 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:44.368983 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:44.369184 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:44.369210 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.369238 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:44.369261 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:44.372351 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.196:36343
I20251209 03:03:44.372421 5580 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.196:36343 every 8 connection(s)
I20251209 03:03:44.375181 5581 heartbeater.cc:344] Connected to a master server at 127.2.170.254:43379
I20251209 03:03:44.375239 5581 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:44.375341 5581 heartbeater.cc:507] Master 127.2.170.254:43379 requested a full tablet report, sending...
I20251209 03:03:44.375538 5226 ts_manager.cc:194] Registered new tserver with Master: c932fb0decca4ccc84aa06706332a26d (127.2.170.196:36343)
I20251209 03:03:44.375808 2731 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.003295678s
I20251209 03:03:44.375869 5226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52096
I20251209 03:03:45.313951 5355 heartbeater.cc:499] Master 127.2.170.254:43379 was elected leader, sending a full tablet report...
I20251209 03:03:45.334715 5431 heartbeater.cc:499] Master 127.2.170.254:43379 was elected leader, sending a full tablet report...
I20251209 03:03:45.357210 5506 heartbeater.cc:499] Master 127.2.170.254:43379 was elected leader, sending a full tablet report...
I20251209 03:03:45.376377 5581 heartbeater.cc:499] Master 127.2.170.254:43379 was elected leader, sending a full tablet report...
I20251209 03:03:45.403810 2731 test_util.cc:276] Using random seed: 686878623
I20251209 03:03:45.408270 5226 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52110:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251209 03:03:45.413079 5319 tablet_service.cc:1505] Processing CreateTablet for tablet c3af3333759c4d94945039800131a95d (DEFAULT_TABLE table=test-workload [id=722f1ae289f94075bf27335627a0e57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:03:45.413244 5319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3af3333759c4d94945039800131a95d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:45.413380 5545 tablet_service.cc:1505] Processing CreateTablet for tablet c3af3333759c4d94945039800131a95d (DEFAULT_TABLE table=test-workload [id=722f1ae289f94075bf27335627a0e57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:03:45.413465 5545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3af3333759c4d94945039800131a95d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:45.413888 5395 tablet_service.cc:1505] Processing CreateTablet for tablet c3af3333759c4d94945039800131a95d (DEFAULT_TABLE table=test-workload [id=722f1ae289f94075bf27335627a0e57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:03:45.414007 5395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3af3333759c4d94945039800131a95d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:45.414777 5602 tablet_bootstrap.cc:492] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c: Bootstrap starting.
I20251209 03:03:45.415028 5604 tablet_bootstrap.cc:492] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e: Bootstrap starting.
I20251209 03:03:45.415180 5602 tablet_bootstrap.cc:654] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:45.415526 5603 tablet_bootstrap.cc:492] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d: Bootstrap starting.
I20251209 03:03:45.415560 5604 tablet_bootstrap.cc:654] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:45.415732 5602 tablet_bootstrap.cc:492] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c: No bootstrap required, opened a new log
I20251209 03:03:45.415791 5602 ts_tablet_manager.cc:1403] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:45.415938 5603 tablet_bootstrap.cc:654] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:45.415943 5602 raft_consensus.cc:359] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.416054 5602 raft_consensus.cc:385] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:45.416077 5602 raft_consensus.cc:740] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e682ab9ef4343cbaef1052beeb1e23c, State: Initialized, Role: FOLLOWER
I20251209 03:03:45.416129 5602 consensus_queue.cc:260] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.416179 5604 tablet_bootstrap.cc:492] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e: No bootstrap required, opened a new log
I20251209 03:03:45.416217 5604 ts_tablet_manager.cc:1403] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:45.416280 5602 ts_tablet_manager.cc:1434] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:03:45.416353 5604 raft_consensus.cc:359] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.416440 5604 raft_consensus.cc:385] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:45.416477 5604 raft_consensus.cc:740] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 865a28131ef54f91b27098e9c848d67e, State: Initialized, Role: FOLLOWER
I20251209 03:03:45.416534 5604 consensus_queue.cc:260] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.416692 5604 ts_tablet_manager.cc:1434] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:03:45.417073 5603 tablet_bootstrap.cc:492] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d: No bootstrap required, opened a new log
I20251209 03:03:45.417150 5603 ts_tablet_manager.cc:1403] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:03:45.417260 5603 raft_consensus.cc:359] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.417299 5603 raft_consensus.cc:385] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:45.417310 5603 raft_consensus.cc:740] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c932fb0decca4ccc84aa06706332a26d, State: Initialized, Role: FOLLOWER
I20251209 03:03:45.417349 5603 consensus_queue.cc:260] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.417472 5603 ts_tablet_manager.cc:1434] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:45.538760 5610 raft_consensus.cc:493] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:03:45.538866 5610 raft_consensus.cc:515] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.539096 5610 leader_election.cc:290] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 865a28131ef54f91b27098e9c848d67e (127.2.170.194:43457), 7e682ab9ef4343cbaef1052beeb1e23c (127.2.170.193:35925)
I20251209 03:03:45.541786 5329 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3af3333759c4d94945039800131a95d" candidate_uuid: "c932fb0decca4ccc84aa06706332a26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" is_pre_election: true
I20251209 03:03:45.541790 5405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3af3333759c4d94945039800131a95d" candidate_uuid: "c932fb0decca4ccc84aa06706332a26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "865a28131ef54f91b27098e9c848d67e" is_pre_election: true
I20251209 03:03:45.541896 5329 raft_consensus.cc:2468] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c932fb0decca4ccc84aa06706332a26d in term 0.
I20251209 03:03:45.541944 5405 raft_consensus.cc:2468] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c932fb0decca4ccc84aa06706332a26d in term 0.
I20251209 03:03:45.542078 5518 leader_election.cc:304] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e682ab9ef4343cbaef1052beeb1e23c, c932fb0decca4ccc84aa06706332a26d; no voters:
I20251209 03:03:45.542172 5610 raft_consensus.cc:2804] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:03:45.542207 5610 raft_consensus.cc:493] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:03:45.542222 5610 raft_consensus.cc:3060] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:45.542829 5610 raft_consensus.cc:515] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.542941 5610 leader_election.cc:290] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [CANDIDATE]: Term 1 election: Requested vote from peers 865a28131ef54f91b27098e9c848d67e (127.2.170.194:43457), 7e682ab9ef4343cbaef1052beeb1e23c (127.2.170.193:35925)
I20251209 03:03:45.543133 5405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3af3333759c4d94945039800131a95d" candidate_uuid: "c932fb0decca4ccc84aa06706332a26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "865a28131ef54f91b27098e9c848d67e"
I20251209 03:03:45.543145 5329 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3af3333759c4d94945039800131a95d" candidate_uuid: "c932fb0decca4ccc84aa06706332a26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e682ab9ef4343cbaef1052beeb1e23c"
I20251209 03:03:45.543226 5405 raft_consensus.cc:3060] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:45.543233 5329 raft_consensus.cc:3060] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:45.543922 5405 raft_consensus.cc:2468] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c932fb0decca4ccc84aa06706332a26d in term 1.
I20251209 03:03:45.543927 5329 raft_consensus.cc:2468] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c932fb0decca4ccc84aa06706332a26d in term 1.
I20251209 03:03:45.544098 5521 leader_election.cc:304] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 865a28131ef54f91b27098e9c848d67e, c932fb0decca4ccc84aa06706332a26d; no voters:
I20251209 03:03:45.544219 5610 raft_consensus.cc:2804] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:03:45.544358 5610 raft_consensus.cc:697] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 1 LEADER]: Becoming Leader. State: Replica: c932fb0decca4ccc84aa06706332a26d, State: Running, Role: LEADER
I20251209 03:03:45.544405 5610 consensus_queue.cc:237] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } }
I20251209 03:03:45.544956 5226 catalog_manager.cc:5654] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d reported cstate change: term changed from 0 to 1, leader changed from <none> to c932fb0decca4ccc84aa06706332a26d (127.2.170.196). New cstate: current_term: 1 leader_uuid: "c932fb0decca4ccc84aa06706332a26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c932fb0decca4ccc84aa06706332a26d" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36343 } health_report { overall_health: HEALTHY } } }
W20251209 03:03:45.559662 5356 tablet.cc:2378] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251209 03:03:46.081255 5615 consensus_queue.cc:1048] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e682ab9ef4343cbaef1052beeb1e23c" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 35925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:03:46.082099 5615 consensus_queue.cc:1048] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [LEADER]: Connected to new peer: Peer: permanent_uuid: "865a28131ef54f91b27098e9c848d67e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 43457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:03:46.260433 5281 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:46.260565 5281 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:46.260596 5281 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:48.260751 5281 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:48.260872 5281 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:48.260893 5281 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:50.261041 5281 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:50.261173 5281 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:50.261209 5281 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:52.261358 5281 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:52.261485 5281 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:52.261502 5281 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:54.261710 5281 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:54.261839 5281 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:54.261858 5281 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:56.262053 5281 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:56.262181 5281 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:56.262228 5281 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:56.604663 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:03:56.606981 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:56.607115 2731 tablet_replica.cc:333] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c: stopping tablet replica
I20251209 03:03:56.607172 2731 raft_consensus.cc:2243] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:03:56.607216 2731 raft_consensus.cc:2272] T c3af3333759c4d94945039800131a95d P 7e682ab9ef4343cbaef1052beeb1e23c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:56.618927 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:03:56.619838 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:03:56.622344 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:56.622473 2731 tablet_replica.cc:333] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e: stopping tablet replica
I20251209 03:03:56.622566 2731 raft_consensus.cc:2243] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:03:56.622618 2731 raft_consensus.cc:2272] T c3af3333759c4d94945039800131a95d P 865a28131ef54f91b27098e9c848d67e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:56.634331 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:03:56.635418 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:03:56.637820 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:56.649170 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:03:56.650110 2731 tablet_server.cc:178] TabletServer@127.2.170.196:0 shutting down...
I20251209 03:03:56.652387 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:03:56.652486 2731 tablet_replica.cc:333] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d: stopping tablet replica
I20251209 03:03:56.652544 2731 raft_consensus.cc:2243] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:03:56.652606 2731 raft_consensus.cc:2272] T c3af3333759c4d94945039800131a95d P c932fb0decca4ccc84aa06706332a26d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:56.664098 2731 tablet_server.cc:195] TabletServer@127.2.170.196:0 shutdown complete.
I20251209 03:03:56.664961 2731 master.cc:561] Master@127.2.170.254:43379 shutting down...
I20251209 03:03:56.667310 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:03:56.667389 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:03:56.667408 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P f571cca2e4744e12a5425c2aa6e8c59e: stopping tablet replica
I20251209 03:03:56.679244 2731 master.cc:583] Master@127.2.170.254:43379 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfLocationLoadSkewedByOne (12443 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy
I20251209 03:03:56.683475 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:40643
I20251209 03:03:56.683657 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:56.685353 5629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:56.685453 5631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:56.685407 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:56.685699 5628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:56.685873 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:56.685910 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:56.685921 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249436685921 us; error 0 us; skew 500 ppm
I20251209 03:03:56.686460 2731 webserver.cc:492] Webserver started at http://127.2.170.254:34073/ using document root <none> and password file <none>
I20251209 03:03:56.686530 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:56.686585 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:56.686626 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:56.686892 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "4be56247af4845f7a9543384f814598e"
format_stamp: "Formatted at 2025-12-09 03:03:56 on dist-test-slave-4cb0"
I20251209 03:03:56.687681 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:56.688059 5636 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.688184 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.688242 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "4be56247af4845f7a9543384f814598e"
format_stamp: "Formatted at 2025-12-09 03:03:56 on dist-test-slave-4cb0"
I20251209 03:03:56.688277 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:56.709594 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:56.709857 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:56.712414 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:40643
I20251209 03:03:56.714325 5688 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:40643 every 8 connection(s)
I20251209 03:03:56.714468 5689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:56.715523 5689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e: Bootstrap starting.
I20251209 03:03:56.715865 5689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:56.716367 5689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e: No bootstrap required, opened a new log
I20251209 03:03:56.716487 5689 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be56247af4845f7a9543384f814598e" member_type: VOTER }
I20251209 03:03:56.716560 5689 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:56.716575 5689 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be56247af4845f7a9543384f814598e, State: Initialized, Role: FOLLOWER
I20251209 03:03:56.716614 5689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be56247af4845f7a9543384f814598e" member_type: VOTER }
I20251209 03:03:56.716651 5689 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:03:56.716668 5689 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:03:56.716693 5689 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:56.717206 5689 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be56247af4845f7a9543384f814598e" member_type: VOTER }
I20251209 03:03:56.717259 5689 leader_election.cc:304] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4be56247af4845f7a9543384f814598e; no voters:
I20251209 03:03:56.717353 5689 leader_election.cc:290] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:03:56.717409 5692 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:03:56.717517 5689 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:03:56.717557 5692 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 1 LEADER]: Becoming Leader. State: Replica: 4be56247af4845f7a9543384f814598e, State: Running, Role: LEADER
I20251209 03:03:56.717636 5692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be56247af4845f7a9543384f814598e" member_type: VOTER }
I20251209 03:03:56.717851 5694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4be56247af4845f7a9543384f814598e. Latest consensus state: current_term: 1 leader_uuid: "4be56247af4845f7a9543384f814598e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be56247af4845f7a9543384f814598e" member_type: VOTER } }
I20251209 03:03:56.717851 5693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4be56247af4845f7a9543384f814598e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4be56247af4845f7a9543384f814598e" member_type: VOTER } }
I20251209 03:03:56.717955 5694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [sys.catalog]: This master's current role is: LEADER
I20251209 03:03:56.717983 5693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [sys.catalog]: This master's current role is: LEADER
I20251209 03:03:56.718232 5699 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:03:56.718358 5699 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:03:56.718961 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:03:56.719063 5699 catalog_manager.cc:1357] Generated new cluster ID: 0d48e491f6c545eba7f5da11a7f393be
I20251209 03:03:56.719111 5699 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:03:56.733770 5699 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:03:56.734247 5699 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:03:56.741458 5699 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e: Generated new TSK 0
I20251209 03:03:56.741559 5699 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:03:56.750857 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:56.752141 5711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:56.752173 5710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:56.752252 5713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:56.752370 2731 server_base.cc:1047] running on GCE node
I20251209 03:03:56.752472 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:56.752504 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:56.752526 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249436752527 us; error 0 us; skew 500 ppm
I20251209 03:03:56.753093 2731 webserver.cc:492] Webserver started at http://127.2.170.193:46449/ using document root <none> and password file <none>
I20251209 03:03:56.753167 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:56.753203 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:56.753238 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:56.753490 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "f3676ba90eed4bd6a1ad254e734151db"
format_stamp: "Formatted at 2025-12-09 03:03:56 on dist-test-slave-4cb0"
I20251209 03:03:56.754350 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:03:56.754750 5718 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.754889 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.754932 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "f3676ba90eed4bd6a1ad254e734151db"
format_stamp: "Formatted at 2025-12-09 03:03:56 on dist-test-slave-4cb0"
I20251209 03:03:56.754977 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:56.769565 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:56.769811 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:56.769956 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:56.770128 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:56.770156 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.770182 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:56.770206 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.772922 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:39079
I20251209 03:03:56.772974 5781 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:39079 every 8 connection(s)
I20251209 03:03:56.773288 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:56.774598 5787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:56.774641 5786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:56.774667 5789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:56.775089 2731 server_base.cc:1047] running on GCE node
I20251209 03:03:56.775193 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:56.775224 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:56.775245 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249436775246 us; error 0 us; skew 500 ppm
I20251209 03:03:56.775796 2731 webserver.cc:492] Webserver started at http://127.2.170.194:40257/ using document root <none> and password file <none>
I20251209 03:03:56.775879 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:56.775925 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:56.775971 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:56.776259 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "03ce7328f2524fac8a913bc2f2130dfd"
format_stamp: "Formatted at 2025-12-09 03:03:56 on dist-test-slave-4cb0"
I20251209 03:03:56.776901 5782 heartbeater.cc:344] Connected to a master server at 127.2.170.254:40643
I20251209 03:03:56.776960 5782 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:56.777068 5782 heartbeater.cc:507] Master 127.2.170.254:40643 requested a full tablet report, sending...
I20251209 03:03:56.777271 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:03:56.777309 5653 ts_manager.cc:194] Registered new tserver with Master: f3676ba90eed4bd6a1ad254e734151db (127.2.170.193:39079)
I20251209 03:03:56.777799 5794 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.777930 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.777967 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "03ce7328f2524fac8a913bc2f2130dfd"
format_stamp: "Formatted at 2025-12-09 03:03:56 on dist-test-slave-4cb0"
I20251209 03:03:56.778012 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:56.778088 5653 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56584
I20251209 03:03:56.785097 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:56.785295 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:56.785460 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:56.785662 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:56.785722 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.785754 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:56.785778 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.788452 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:35621
I20251209 03:03:56.788499 5857 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:35621 every 8 connection(s)
I20251209 03:03:56.788800 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:03:56.790104 5861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:03:56.790196 5862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:56.790236 2731 server_base.cc:1047] running on GCE node
W20251209 03:03:56.790339 5864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:03:56.790524 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:03:56.790555 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:03:56.790575 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249436790575 us; error 0 us; skew 500 ppm
I20251209 03:03:56.791164 2731 webserver.cc:492] Webserver started at http://127.2.170.195:45969/ using document root <none> and password file <none>
I20251209 03:03:56.791249 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:03:56.791291 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:03:56.791335 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:03:56.791640 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "db13d6019f064898a88e86c9664369fc"
format_stamp: "Formatted at 2025-12-09 03:03:56 on dist-test-slave-4cb0"
I20251209 03:03:56.792032 5858 heartbeater.cc:344] Connected to a master server at 127.2.170.254:40643
I20251209 03:03:56.792093 5858 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:56.792187 5858 heartbeater.cc:507] Master 127.2.170.254:40643 requested a full tablet report, sending...
I20251209 03:03:56.792379 5653 ts_manager.cc:194] Registered new tserver with Master: 03ce7328f2524fac8a913bc2f2130dfd (127.2.170.194:35621)
I20251209 03:03:56.792625 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:56.792709 5653 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56590
I20251209 03:03:56.793103 5869 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.793213 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.793258 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "db13d6019f064898a88e86c9664369fc"
format_stamp: "Formatted at 2025-12-09 03:03:56 on dist-test-slave-4cb0"
I20251209 03:03:56.793301 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:03:56.810173 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:03:56.810426 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:03:56.810595 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:03:56.810787 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:03:56.810814 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.810843 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:03:56.810864 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:56.813934 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:43051
I20251209 03:03:56.813980 5932 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:43051 every 8 connection(s)
I20251209 03:03:56.817123 5933 heartbeater.cc:344] Connected to a master server at 127.2.170.254:40643
I20251209 03:03:56.817186 5933 heartbeater.cc:461] Registering TS with master...
I20251209 03:03:56.817286 5933 heartbeater.cc:507] Master 127.2.170.254:40643 requested a full tablet report, sending...
I20251209 03:03:56.817541 5653 ts_manager.cc:194] Registered new tserver with Master: db13d6019f064898a88e86c9664369fc (127.2.170.195:43051)
I20251209 03:03:56.817925 5653 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56596
I20251209 03:03:56.818497 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004384259s
I20251209 03:03:57.778623 5782 heartbeater.cc:499] Master 127.2.170.254:40643 was elected leader, sending a full tablet report...
I20251209 03:03:57.793223 5858 heartbeater.cc:499] Master 127.2.170.254:40643 was elected leader, sending a full tablet report...
I20251209 03:03:57.818456 5933 heartbeater.cc:499] Master 127.2.170.254:40643 was elected leader, sending a full tablet report...
I20251209 03:03:57.846944 2731 test_util.cc:276] Using random seed: 699321756
I20251209 03:03:57.851890 5653 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56628:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:03:57.852026 5653 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:03:57.856880 5822 tablet_service.cc:1505] Processing CreateTablet for tablet ff2695e0c6484d6f9f90ac576d02a222 (DEFAULT_TABLE table=test-workload [id=bef89aa05cd748768f3b7318340c9b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:03:57.857025 5822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2695e0c6484d6f9f90ac576d02a222. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:57.857038 5746 tablet_service.cc:1505] Processing CreateTablet for tablet ff2695e0c6484d6f9f90ac576d02a222 (DEFAULT_TABLE table=test-workload [id=bef89aa05cd748768f3b7318340c9b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:03:57.857153 5746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2695e0c6484d6f9f90ac576d02a222. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:57.857507 5897 tablet_service.cc:1505] Processing CreateTablet for tablet ff2695e0c6484d6f9f90ac576d02a222 (DEFAULT_TABLE table=test-workload [id=bef89aa05cd748768f3b7318340c9b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:03:57.857604 5897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2695e0c6484d6f9f90ac576d02a222. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:03:57.858449 5954 tablet_bootstrap.cc:492] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db: Bootstrap starting.
I20251209 03:03:57.858664 5955 tablet_bootstrap.cc:492] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd: Bootstrap starting.
I20251209 03:03:57.858956 5954 tablet_bootstrap.cc:654] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:57.859248 5955 tablet_bootstrap.cc:654] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:57.859329 5956 tablet_bootstrap.cc:492] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc: Bootstrap starting.
I20251209 03:03:57.859534 5954 tablet_bootstrap.cc:492] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db: No bootstrap required, opened a new log
I20251209 03:03:57.859579 5954 ts_tablet_manager.cc:1403] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:57.859731 5954 raft_consensus.cc:359] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:57.859760 5955 tablet_bootstrap.cc:492] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd: No bootstrap required, opened a new log
I20251209 03:03:57.859797 5954 raft_consensus.cc:385] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:57.859817 5955 ts_tablet_manager.cc:1403] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:03:57.859845 5954 raft_consensus.cc:740] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3676ba90eed4bd6a1ad254e734151db, State: Initialized, Role: FOLLOWER
I20251209 03:03:57.859890 5954 consensus_queue.cc:260] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:57.859956 5955 raft_consensus.cc:359] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:57.860018 5955 raft_consensus.cc:385] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:57.860038 5955 raft_consensus.cc:740] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ce7328f2524fac8a913bc2f2130dfd, State: Initialized, Role: FOLLOWER
I20251209 03:03:57.860069 5954 ts_tablet_manager.cc:1434] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:03:57.860100 5955 consensus_queue.cc:260] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:57.860261 5955 ts_tablet_manager.cc:1434] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:57.860560 5956 tablet_bootstrap.cc:654] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc: Neither blocks nor log segments found. Creating new log.
I20251209 03:03:57.861155 5956 tablet_bootstrap.cc:492] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc: No bootstrap required, opened a new log
I20251209 03:03:57.861194 5956 ts_tablet_manager.cc:1403] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:03:57.861308 5956 raft_consensus.cc:359] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:57.861352 5956 raft_consensus.cc:385] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:03:57.861364 5956 raft_consensus.cc:740] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db13d6019f064898a88e86c9664369fc, State: Initialized, Role: FOLLOWER
I20251209 03:03:57.861402 5956 consensus_queue.cc:260] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:57.861531 5956 ts_tablet_manager.cc:1434] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:03:58.039474 5959 raft_consensus.cc:493] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:03:58.039608 5959 raft_consensus.cc:515] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:58.039834 5959 leader_election.cc:290] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03ce7328f2524fac8a913bc2f2130dfd (127.2.170.194:35621), db13d6019f064898a88e86c9664369fc (127.2.170.195:43051)
I20251209 03:03:58.042461 5907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff2695e0c6484d6f9f90ac576d02a222" candidate_uuid: "f3676ba90eed4bd6a1ad254e734151db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db13d6019f064898a88e86c9664369fc" is_pre_election: true
I20251209 03:03:58.042512 5832 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff2695e0c6484d6f9f90ac576d02a222" candidate_uuid: "f3676ba90eed4bd6a1ad254e734151db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03ce7328f2524fac8a913bc2f2130dfd" is_pre_election: true
I20251209 03:03:58.042557 5907 raft_consensus.cc:2468] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3676ba90eed4bd6a1ad254e734151db in term 0.
I20251209 03:03:58.042611 5832 raft_consensus.cc:2468] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3676ba90eed4bd6a1ad254e734151db in term 0.
I20251209 03:03:58.042776 5722 leader_election.cc:304] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db13d6019f064898a88e86c9664369fc, f3676ba90eed4bd6a1ad254e734151db; no voters:
I20251209 03:03:58.042871 5959 raft_consensus.cc:2804] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:03:58.042910 5959 raft_consensus.cc:493] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:03:58.042925 5959 raft_consensus.cc:3060] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:58.043551 5959 raft_consensus.cc:515] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:58.043671 5959 leader_election.cc:290] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [CANDIDATE]: Term 1 election: Requested vote from peers 03ce7328f2524fac8a913bc2f2130dfd (127.2.170.194:35621), db13d6019f064898a88e86c9664369fc (127.2.170.195:43051)
I20251209 03:03:58.043838 5832 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff2695e0c6484d6f9f90ac576d02a222" candidate_uuid: "f3676ba90eed4bd6a1ad254e734151db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03ce7328f2524fac8a913bc2f2130dfd"
I20251209 03:03:58.043871 5907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff2695e0c6484d6f9f90ac576d02a222" candidate_uuid: "f3676ba90eed4bd6a1ad254e734151db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db13d6019f064898a88e86c9664369fc"
I20251209 03:03:58.043898 5832 raft_consensus.cc:3060] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:58.043915 5907 raft_consensus.cc:3060] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:03:58.044513 5832 raft_consensus.cc:2468] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3676ba90eed4bd6a1ad254e734151db in term 1.
I20251209 03:03:58.044519 5907 raft_consensus.cc:2468] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3676ba90eed4bd6a1ad254e734151db in term 1.
I20251209 03:03:58.044726 5720 leader_election.cc:304] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03ce7328f2524fac8a913bc2f2130dfd, f3676ba90eed4bd6a1ad254e734151db; no voters:
I20251209 03:03:58.044842 5959 raft_consensus.cc:2804] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:03:58.044991 5959 raft_consensus.cc:697] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 1 LEADER]: Becoming Leader. State: Replica: f3676ba90eed4bd6a1ad254e734151db, State: Running, Role: LEADER
I20251209 03:03:58.045037 5959 consensus_queue.cc:237] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } }
I20251209 03:03:58.045557 5652 catalog_manager.cc:5654] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db reported cstate change: term changed from 0 to 1, leader changed from <none> to f3676ba90eed4bd6a1ad254e734151db (127.2.170.193). New cstate: current_term: 1 leader_uuid: "f3676ba90eed4bd6a1ad254e734151db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3676ba90eed4bd6a1ad254e734151db" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 } health_report { overall_health: UNKNOWN } } }
I20251209 03:03:58.539783 5959 consensus_queue.cc:1048] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [LEADER]: Connected to new peer: Peer: permanent_uuid: "03ce7328f2524fac8a913bc2f2130dfd" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 35621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:03:58.542507 5965 consensus_queue.cc:1048] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [LEADER]: Connected to new peer: Peer: permanent_uuid: "db13d6019f064898a88e86c9664369fc" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 43051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:03:58.718703 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:03:58.718858 5707 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:03:58.718931 5707 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:03:58.718964 5707 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:03:59.719274 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:00.719105 5707 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:00.719228 5707 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:00.719254 5707 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:00.719648 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:01.719960 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:02.719403 5707 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:02.719545 5707 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:02.719564 5707 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:02.720341 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:03.720716 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:04.719726 5707 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:04.719846 5707 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:04.719870 5707 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:04.721030 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:05.721459 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:06.720011 5707 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:06.720140 5707 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:06.720165 5707 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:06.721791 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:07.722098 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:08.720331 5707 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:08.720470 5707 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:08.720497 5707 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:08.722461 5702 placement_policy_util.cc:407] tablet ff2695e0c6484d6f9f90ac576d02a222: detected majority of replicas (2 of 3) at location L0
I20251209 03:04:09.129931 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:04:09.132263 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:09.132449 2731 tablet_replica.cc:333] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db: stopping tablet replica
I20251209 03:04:09.132501 2731 raft_consensus.cc:2243] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:04:09.132589 2731 raft_consensus.cc:2272] T ff2695e0c6484d6f9f90ac576d02a222 P f3676ba90eed4bd6a1ad254e734151db [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:09.144376 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:04:09.145267 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:04:09.147723 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:09.147847 2731 tablet_replica.cc:333] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd: stopping tablet replica
I20251209 03:04:09.147917 2731 raft_consensus.cc:2243] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:09.147964 2731 raft_consensus.cc:2272] T ff2695e0c6484d6f9f90ac576d02a222 P 03ce7328f2524fac8a913bc2f2130dfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:09.159319 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:04:09.160068 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:04:09.162236 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:09.162398 2731 tablet_replica.cc:333] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc: stopping tablet replica
I20251209 03:04:09.162447 2731 raft_consensus.cc:2243] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:09.162495 2731 raft_consensus.cc:2272] T ff2695e0c6484d6f9f90ac576d02a222 P db13d6019f064898a88e86c9664369fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:09.173841 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:04:09.174666 2731 master.cc:561] Master@127.2.170.254:40643 shutting down...
I20251209 03:04:09.177419 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:04:09.177505 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:09.177534 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4be56247af4845f7a9543384f814598e: stopping tablet replica
I20251209 03:04:09.189186 2731 master.cc:583] Master@127.2.170.254:40643 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotFixPlacementPolicy (12509 ms)
[ RUN ] AutoRebalancerTest.TestMaxMovesPerServer
I20251209 03:04:09.192957 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:39481
I20251209 03:04:09.193099 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:09.194422 5979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:09.194510 5977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:09.194530 5976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:09.194664 2731 server_base.cc:1047] running on GCE node
I20251209 03:04:09.194743 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:09.194773 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:09.194795 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249449194795 us; error 0 us; skew 500 ppm
I20251209 03:04:09.195458 2731 webserver.cc:492] Webserver started at http://127.2.170.254:40069/ using document root <none> and password file <none>
I20251209 03:04:09.195533 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:09.195576 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:09.195619 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:09.195859 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "6f5640f467eb4d63bcca994b2fa93dd8"
format_stamp: "Formatted at 2025-12-09 03:04:09 on dist-test-slave-4cb0"
I20251209 03:04:09.196632 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:09.197036 5984 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.197145 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.197185 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "6f5640f467eb4d63bcca994b2fa93dd8"
format_stamp: "Formatted at 2025-12-09 03:04:09 on dist-test-slave-4cb0"
I20251209 03:04:09.197229 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:09.226228 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:09.226433 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:09.228842 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:39481
I20251209 03:04:09.230547 6036 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:39481 every 8 connection(s)
I20251209 03:04:09.230711 6037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:09.232156 6037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8: Bootstrap starting.
I20251209 03:04:09.232496 6037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:09.233014 6037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8: No bootstrap required, opened a new log
I20251209 03:04:09.233150 6037 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5640f467eb4d63bcca994b2fa93dd8" member_type: VOTER }
I20251209 03:04:09.233211 6037 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:09.233229 6037 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f5640f467eb4d63bcca994b2fa93dd8, State: Initialized, Role: FOLLOWER
I20251209 03:04:09.233305 6037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5640f467eb4d63bcca994b2fa93dd8" member_type: VOTER }
I20251209 03:04:09.233351 6037 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:04:09.233378 6037 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:04:09.233407 6037 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:09.234037 6037 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5640f467eb4d63bcca994b2fa93dd8" member_type: VOTER }
I20251209 03:04:09.234110 6037 leader_election.cc:304] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f5640f467eb4d63bcca994b2fa93dd8; no voters:
I20251209 03:04:09.234216 6037 leader_election.cc:290] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:04:09.234263 6040 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:09.234354 6037 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:04:09.234416 6040 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 1 LEADER]: Becoming Leader. State: Replica: 6f5640f467eb4d63bcca994b2fa93dd8, State: Running, Role: LEADER
I20251209 03:04:09.234470 6040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5640f467eb4d63bcca994b2fa93dd8" member_type: VOTER }
I20251209 03:04:09.234711 6040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f5640f467eb4d63bcca994b2fa93dd8. Latest consensus state: current_term: 1 leader_uuid: "6f5640f467eb4d63bcca994b2fa93dd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5640f467eb4d63bcca994b2fa93dd8" member_type: VOTER } }
I20251209 03:04:09.234776 6040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [sys.catalog]: This master's current role is: LEADER
I20251209 03:04:09.234925 6046 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:04:09.234720 6042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f5640f467eb4d63bcca994b2fa93dd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5640f467eb4d63bcca994b2fa93dd8" member_type: VOTER } }
I20251209 03:04:09.235064 6046 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:04:09.235128 6042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [sys.catalog]: This master's current role is: LEADER
I20251209 03:04:09.235623 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:04:09.236086 6046 catalog_manager.cc:1357] Generated new cluster ID: ca6e0a4a49054710b8a4e262249e1932
I20251209 03:04:09.236130 6046 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:04:09.243650 6046 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:04:09.244040 6046 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:04:09.260608 6046 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8: Generated new TSK 0
I20251209 03:04:09.260716 6046 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:04:09.267372 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:09.268657 6058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:09.268816 2731 server_base.cc:1047] running on GCE node
W20251209 03:04:09.268865 6059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:09.269001 6061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:09.269130 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:09.269161 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:09.269172 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249449269173 us; error 0 us; skew 500 ppm
I20251209 03:04:09.269711 2731 webserver.cc:492] Webserver started at http://127.2.170.193:42447/ using document root <none> and password file <none>
I20251209 03:04:09.269797 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:09.269842 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:09.269879 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:09.270089 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "d36845214167428eb93883ecbe8f5385"
format_stamp: "Formatted at 2025-12-09 03:04:09 on dist-test-slave-4cb0"
I20251209 03:04:09.270826 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:09.271189 6066 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.271301 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.271342 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "d36845214167428eb93883ecbe8f5385"
format_stamp: "Formatted at 2025-12-09 03:04:09 on dist-test-slave-4cb0"
I20251209 03:04:09.271377 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:09.280239 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:09.280390 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:09.280509 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:09.280647 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:09.280671 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.280696 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:09.280712 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.283341 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:43899
I20251209 03:04:09.283370 6129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:43899 every 8 connection(s)
I20251209 03:04:09.283628 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:09.284639 6137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:09.284698 6134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:09.284751 6135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:09.284897 2731 server_base.cc:1047] running on GCE node
I20251209 03:04:09.285000 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:09.285032 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:09.285048 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249449285049 us; error 0 us; skew 500 ppm
I20251209 03:04:09.285693 2731 webserver.cc:492] Webserver started at http://127.2.170.194:36941/ using document root <none> and password file <none>
I20251209 03:04:09.285776 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:09.285818 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:09.285861 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:09.286123 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
format_stamp: "Formatted at 2025-12-09 03:04:09 on dist-test-slave-4cb0"
I20251209 03:04:09.287078 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:09.287079 6130 heartbeater.cc:344] Connected to a master server at 127.2.170.254:39481
I20251209 03:04:09.287163 6130 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:09.287267 6130 heartbeater.cc:507] Master 127.2.170.254:39481 requested a full tablet report, sending...
I20251209 03:04:09.287485 6001 ts_manager.cc:194] Registered new tserver with Master: d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899)
I20251209 03:04:09.287602 6142 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.287725 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.287763 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
format_stamp: "Formatted at 2025-12-09 03:04:09 on dist-test-slave-4cb0"
I20251209 03:04:09.287804 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:09.288256 6001 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53134
I20251209 03:04:09.306983 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:09.307171 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:09.307298 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:09.307492 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:09.307520 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.307546 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:09.307569 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.310166 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:42307
I20251209 03:04:09.310213 6205 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:42307 every 8 connection(s)
I20251209 03:04:09.310505 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:09.311635 6210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:09.311681 6212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:09.311859 2731 server_base.cc:1047] running on GCE node
W20251209 03:04:09.311767 6209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:09.312134 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:09.312170 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:09.312186 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249449312186 us; error 0 us; skew 500 ppm
I20251209 03:04:09.312804 2731 webserver.cc:492] Webserver started at http://127.2.170.195:35505/ using document root <none> and password file <none>
I20251209 03:04:09.312883 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:09.312927 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:09.312969 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:09.313238 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "b2fc3e20641e45a4b62732385fd4e036"
format_stamp: "Formatted at 2025-12-09 03:04:09 on dist-test-slave-4cb0"
I20251209 03:04:09.314018 6206 heartbeater.cc:344] Connected to a master server at 127.2.170.254:39481
I20251209 03:04:09.314076 6206 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:09.314179 6206 heartbeater.cc:507] Master 127.2.170.254:39481 requested a full tablet report, sending...
I20251209 03:04:09.314433 6001 ts_manager.cc:194] Registered new tserver with Master: a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:09.314754 6001 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53140
I20251209 03:04:09.314942 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20251209 03:04:09.315553 6217 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.315696 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251209 03:04:09.315747 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "b2fc3e20641e45a4b62732385fd4e036"
format_stamp: "Formatted at 2025-12-09 03:04:09 on dist-test-slave-4cb0"
I20251209 03:04:09.315793 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:09.330358 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:09.330545 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:09.330668 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:09.330816 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:09.330842 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.330869 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:09.330892 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:09.333882 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:34597
I20251209 03:04:09.333926 6280 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:34597 every 8 connection(s)
I20251209 03:04:09.336549 6281 heartbeater.cc:344] Connected to a master server at 127.2.170.254:39481
I20251209 03:04:09.336596 6281 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:09.336680 6281 heartbeater.cc:507] Master 127.2.170.254:39481 requested a full tablet report, sending...
I20251209 03:04:09.336856 6001 ts_manager.cc:194] Registered new tserver with Master: b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:09.337149 6001 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53150
I20251209 03:04:09.337320 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003279396s
I20251209 03:04:10.288775 6130 heartbeater.cc:499] Master 127.2.170.254:39481 was elected leader, sending a full tablet report...
I20251209 03:04:10.315214 6206 heartbeater.cc:499] Master 127.2.170.254:39481 was elected leader, sending a full tablet report...
I20251209 03:04:10.337564 6281 heartbeater.cc:499] Master 127.2.170.254:39481 was elected leader, sending a full tablet report...
I20251209 03:04:10.362310 2731 test_util.cc:276] Using random seed: 711837124
I20251209 03:04:10.367539 6001 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53166:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu""\004\001\000\252\252\252\n\004\001\000TUU\025\004\001\000\376\377\377\037\004\001\000\250\252\252*\004\001\000RUU5\004\001\000\374\377\377?\004\001\000\246\252\252J\004\001\000PUUU\004\001\000\372\377\377_\004\001\000\244\252\252j\004\001\000NUUu"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:04:10.367729 6001 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:04:10.373461 6170 tablet_service.cc:1505] Processing CreateTablet for tablet 5ef8ec5789d64b8f931598cfcded95f2 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251209 03:04:10.373589 6169 tablet_service.cc:1505] Processing CreateTablet for tablet cf139ecb17be4ec58d5c23e86eab88a4 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251209 03:04:10.373617 6170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ef8ec5789d64b8f931598cfcded95f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.373692 6169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf139ecb17be4ec58d5c23e86eab88a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.373778 6245 tablet_service.cc:1505] Processing CreateTablet for tablet 5ef8ec5789d64b8f931598cfcded95f2 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251209 03:04:10.373827 6243 tablet_service.cc:1505] Processing CreateTablet for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251209 03:04:10.373914 6245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ef8ec5789d64b8f931598cfcded95f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.373978 6242 tablet_service.cc:1505] Processing CreateTablet for tablet f9a52de6742a4e508e06374063561398 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251209 03:04:10.374043 6242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a52de6742a4e508e06374063561398. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.375104 6169 tablet_service.cc:1505] Processing CreateTablet for tablet 86b92235e7ad4a959951bdb816e39905 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251209 03:04:10.375197 6169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b92235e7ad4a959951bdb816e39905. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.375214 6242 tablet_service.cc:1505] Processing CreateTablet for tablet 86b92235e7ad4a959951bdb816e39905 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251209 03:04:10.375288 6245 tablet_service.cc:1505] Processing CreateTablet for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251209 03:04:10.375345 6241 tablet_service.cc:1505] Processing CreateTablet for tablet 2b74f825019240c7950dcc767dd2e6a5 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251209 03:04:10.375427 6236 tablet_service.cc:1505] Processing CreateTablet for tablet c5b028a080314d919052909cd10db307 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251209 03:04:10.375479 6240 tablet_service.cc:1505] Processing CreateTablet for tablet 7ead69a2942346dca25124b075136d12 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251209 03:04:10.373793 6244 tablet_service.cc:1505] Processing CreateTablet for tablet cf139ecb17be4ec58d5c23e86eab88a4 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251209 03:04:10.375298 6302 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.375578 6239 tablet_service.cc:1505] Processing CreateTablet for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251209 03:04:10.375298 6242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b92235e7ad4a959951bdb816e39905. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.375672 6239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.375734 6241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b74f825019240c7950dcc767dd2e6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.376178 6169 tablet_service.cc:1505] Processing CreateTablet for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251209 03:04:10.376250 6169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.376823 6236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b028a080314d919052909cd10db307. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.376847 6238 tablet_service.cc:1505] Processing CreateTablet for tablet 439ea3fec68444aeb8cc383b977a7f48 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251209 03:04:10.376938 6238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 439ea3fec68444aeb8cc383b977a7f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.377306 6302 tablet_bootstrap.cc:654] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.377385 6167 tablet_service.cc:1505] Processing CreateTablet for tablet f9a52de6742a4e508e06374063561398 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251209 03:04:10.377470 6167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a52de6742a4e508e06374063561398. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.377970 6244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf139ecb17be4ec58d5c23e86eab88a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.377993 6237 tablet_service.cc:1505] Processing CreateTablet for tablet f71e8ee2ce1242608207622b267d96ae (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251209 03:04:10.378082 6237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71e8ee2ce1242608207622b267d96ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.378548 6166 tablet_service.cc:1505] Processing CreateTablet for tablet 2b74f825019240c7950dcc767dd2e6a5 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251209 03:04:10.378611 6166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b74f825019240c7950dcc767dd2e6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.379132 6245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.379346 6243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a5e7d1a2de4913a09dfeb084d79ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.379474 6165 tablet_service.cc:1505] Processing CreateTablet for tablet 7ead69a2942346dca25124b075136d12 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251209 03:04:10.379541 6165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ead69a2942346dca25124b075136d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.380391 6164 tablet_service.cc:1505] Processing CreateTablet for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251209 03:04:10.380435 6240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ead69a2942346dca25124b075136d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.380492 6094 tablet_service.cc:1505] Processing CreateTablet for tablet 5ef8ec5789d64b8f931598cfcded95f2 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION VALUES < 178956970
I20251209 03:04:10.380554 6094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ef8ec5789d64b8f931598cfcded95f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.381330 6301 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.381603 6304 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.381662 6163 tablet_service.cc:1505] Processing CreateTablet for tablet 439ea3fec68444aeb8cc383b977a7f48 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251209 03:04:10.381625 6094 tablet_service.cc:1505] Processing CreateTablet for tablet 86b92235e7ad4a959951bdb816e39905 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1789569700 <= VALUES < 1968526670
I20251209 03:04:10.381790 6085 tablet_service.cc:1505] Processing CreateTablet for tablet c5b028a080314d919052909cd10db307 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251209 03:04:10.381846 6094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b92235e7ad4a959951bdb816e39905. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.381914 6086 tablet_service.cc:1505] Processing CreateTablet for tablet f71e8ee2ce1242608207622b267d96ae (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251209 03:04:10.381946 6085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b028a080314d919052909cd10db307. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.382033 6087 tablet_service.cc:1505] Processing CreateTablet for tablet 439ea3fec68444aeb8cc383b977a7f48 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1252698790 <= VALUES < 1431655760
I20251209 03:04:10.382069 6088 tablet_service.cc:1505] Processing CreateTablet for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1252698790
I20251209 03:04:10.382120 6087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 439ea3fec68444aeb8cc383b977a7f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.382172 6089 tablet_service.cc:1505] Processing CreateTablet for tablet 7ead69a2942346dca25124b075136d12 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 894784850 <= VALUES < 1073741820
I20251209 03:04:10.382270 6089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ead69a2942346dca25124b075136d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.382851 6094 tablet_service.cc:1505] Processing CreateTablet for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1968526670 <= VALUES
I20251209 03:04:10.382920 6094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.382031 6304 tablet_bootstrap.cc:654] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.383445 6090 tablet_service.cc:1505] Processing CreateTablet for tablet 2b74f825019240c7950dcc767dd2e6a5 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 715827880 <= VALUES < 894784850
I20251209 03:04:10.383533 6090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b74f825019240c7950dcc767dd2e6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.383893 6304 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.383942 6304 ts_tablet_manager.cc:1403] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:04:10.384194 6304 raft_consensus.cc:359] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.384289 6304 raft_consensus.cc:385] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.384320 6304 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.384373 6304 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.384546 6304 ts_tablet_manager.cc:1434] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.384711 6304 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.384825 6091 tablet_service.cc:1505] Processing CreateTablet for tablet f9a52de6742a4e508e06374063561398 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 715827880
I20251209 03:04:10.384925 6091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a52de6742a4e508e06374063561398. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.385139 6304 tablet_bootstrap.cc:654] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.385985 6304 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.386039 6304 ts_tablet_manager.cc:1403] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.386112 6161 tablet_service.cc:1505] Processing CreateTablet for tablet c5b028a080314d919052909cd10db307 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1789569700
I20251209 03:04:10.386180 6304 raft_consensus.cc:359] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.386237 6092 tablet_service.cc:1505] Processing CreateTablet for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251209 03:04:10.386243 6304 raft_consensus.cc:385] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.386301 6304 raft_consensus.cc:740] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.386328 6092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a5e7d1a2de4913a09dfeb084d79ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.386447 6304 consensus_queue.cc:260] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.386565 6304 ts_tablet_manager.cc:1434] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.386637 6304 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.386376 6162 tablet_service.cc:1505] Processing CreateTablet for tablet f71e8ee2ce1242608207622b267d96ae (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 1431655760 <= VALUES < 1610612730
I20251209 03:04:10.386412 6093 tablet_service.cc:1505] Processing CreateTablet for tablet cf139ecb17be4ec58d5c23e86eab88a4 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 178956970 <= VALUES < 357913940
I20251209 03:04:10.386999 6304 tablet_bootstrap.cc:654] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.387187 6093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf139ecb17be4ec58d5c23e86eab88a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.387934 6304 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.388020 6304 ts_tablet_manager.cc:1403] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.388219 6304 raft_consensus.cc:359] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.388283 6304 raft_consensus.cc:385] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.388303 6304 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.388331 6304 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.377315 6168 tablet_service.cc:1505] Processing CreateTablet for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 (DEFAULT_TABLE table=test-workload [id=f83c37083814476fb11fdb400ba48a2a]), partition=RANGE (key) PARTITION 357913940 <= VALUES < 536870910
I20251209 03:04:10.388550 6088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.388428 6304 ts_tablet_manager.cc:1434] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.388667 6304 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.389112 6304 tablet_bootstrap.cc:654] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.380451 6164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.389895 6163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 439ea3fec68444aeb8cc383b977a7f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.390208 6301 tablet_bootstrap.cc:654] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.390503 6168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a5e7d1a2de4913a09dfeb084d79ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.390874 6086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71e8ee2ce1242608207622b267d96ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.391578 6301 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.391623 6301 ts_tablet_manager.cc:1403] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251209 03:04:10.391754 6161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b028a080314d919052909cd10db307. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.391767 6301 raft_consensus.cc:359] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.391839 6301 raft_consensus.cc:385] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.391861 6301 raft_consensus.cc:740] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.391892 6301 consensus_queue.cc:260] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.392036 6301 ts_tablet_manager.cc:1434] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.392098 6301 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.392480 6301 tablet_bootstrap.cc:654] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.392500 6162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71e8ee2ce1242608207622b267d96ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:10.393025 6301 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.393066 6301 ts_tablet_manager.cc:1403] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.393186 6301 raft_consensus.cc:359] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.393234 6301 raft_consensus.cc:385] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.393250 6301 raft_consensus.cc:740] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.393278 6301 consensus_queue.cc:260] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.393376 6301 ts_tablet_manager.cc:1434] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.393428 6301 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.381662 6302 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.393734 6302 ts_tablet_manager.cc:1403] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.001s
I20251209 03:04:10.393822 6301 tablet_bootstrap.cc:654] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.393867 6302 raft_consensus.cc:359] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.393904 6302 raft_consensus.cc:385] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.393920 6302 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.393956 6302 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.394120 6302 ts_tablet_manager.cc:1434] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.394186 6302 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.394330 6301 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.394390 6301 ts_tablet_manager.cc:1403] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.394541 6302 tablet_bootstrap.cc:654] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.394526 6301 raft_consensus.cc:359] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.394579 6301 raft_consensus.cc:385] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.394608 6301 raft_consensus.cc:740] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.394644 6301 consensus_queue.cc:260] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.394727 6301 ts_tablet_manager.cc:1434] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.394784 6301 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.395058 6302 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.395099 6302 ts_tablet_manager.cc:1403] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.395108 6301 tablet_bootstrap.cc:654] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.395239 6302 raft_consensus.cc:359] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.395284 6302 raft_consensus.cc:385] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.395303 6302 raft_consensus.cc:740] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.395344 6302 consensus_queue.cc:260] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.395423 6302 ts_tablet_manager.cc:1434] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.395476 6302 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.395790 6301 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.395843 6301 ts_tablet_manager.cc:1403] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.395854 6302 tablet_bootstrap.cc:654] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.395990 6301 raft_consensus.cc:359] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.396049 6301 raft_consensus.cc:385] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.396070 6301 raft_consensus.cc:740] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.396104 6301 consensus_queue.cc:260] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.396209 6301 ts_tablet_manager.cc:1434] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.396267 6301 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.396441 6304 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.396477 6304 ts_tablet_manager.cc:1403] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20251209 03:04:10.396631 6301 tablet_bootstrap.cc:654] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.396595 6304 raft_consensus.cc:359] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.396661 6302 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.396720 6302 ts_tablet_manager.cc:1403] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.396667 6304 raft_consensus.cc:385] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.396795 6304 raft_consensus.cc:740] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.396843 6304 consensus_queue.cc:260] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.396878 6302 raft_consensus.cc:359] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.396939 6304 ts_tablet_manager.cc:1434] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:04:10.396946 6302 raft_consensus.cc:385] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.396998 6304 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.397002 6302 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.397078 6302 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.397383 6304 tablet_bootstrap.cc:654] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.397580 6302 ts_tablet_manager.cc:1434] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.397805 6302 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.398072 6301 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.398115 6301 ts_tablet_manager.cc:1403] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:04:10.398149 6302 tablet_bootstrap.cc:654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.398247 6301 raft_consensus.cc:359] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.398303 6301 raft_consensus.cc:385] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.398324 6301 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.398375 6301 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.398460 6301 ts_tablet_manager.cc:1434] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.398561 6301 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.398623 6304 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.398670 6304 ts_tablet_manager.cc:1403] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251209 03:04:10.398770 6302 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.398808 6302 ts_tablet_manager.cc:1403] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.398816 6304 raft_consensus.cc:359] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.398869 6304 raft_consensus.cc:385] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.398886 6304 raft_consensus.cc:740] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.398922 6304 consensus_queue.cc:260] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.398962 6301 tablet_bootstrap.cc:654] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.398993 6304 ts_tablet_manager.cc:1434] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.399060 6304 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.398967 6302 raft_consensus.cc:359] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.399350 6302 raft_consensus.cc:385] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.399377 6302 raft_consensus.cc:740] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.399420 6302 consensus_queue.cc:260] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.399479 6301 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.399506 6302 ts_tablet_manager.cc:1434] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.399526 6301 ts_tablet_manager.cc:1403] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.399566 6302 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.399662 6301 raft_consensus.cc:359] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.399712 6301 raft_consensus.cc:385] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.399724 6301 raft_consensus.cc:740] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.399755 6301 consensus_queue.cc:260] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.399829 6301 ts_tablet_manager.cc:1434] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.399878 6301 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.399930 6302 tablet_bootstrap.cc:654] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.400198 6301 tablet_bootstrap.cc:654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.400473 6302 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.400516 6302 ts_tablet_manager.cc:1403] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.399435 6304 tablet_bootstrap.cc:654] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.400635 6302 raft_consensus.cc:359] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.400697 6302 raft_consensus.cc:385] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.400714 6302 raft_consensus.cc:740] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.400749 6302 consensus_queue.cc:260] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.400825 6302 ts_tablet_manager.cc:1434] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.400874 6302 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.401144 6304 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.401186 6304 ts_tablet_manager.cc:1403] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251209 03:04:10.401187 6302 tablet_bootstrap.cc:654] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.401322 6304 raft_consensus.cc:359] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.401393 6304 raft_consensus.cc:385] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.401408 6304 raft_consensus.cc:740] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.401436 6304 consensus_queue.cc:260] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.401561 6304 ts_tablet_manager.cc:1434] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.401614 6304 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.401818 6302 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.401860 6302 ts_tablet_manager.cc:1403] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.401995 6302 raft_consensus.cc:359] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.402040 6302 raft_consensus.cc:385] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.402043 6304 tablet_bootstrap.cc:654] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.402073 6302 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.402230 6302 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.402304 6302 ts_tablet_manager.cc:1434] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.402359 6302 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.402813 6302 tablet_bootstrap.cc:654] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.403007 6304 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.403049 6304 ts_tablet_manager.cc:1403] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.403175 6304 raft_consensus.cc:359] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.403234 6304 raft_consensus.cc:385] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.403259 6304 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.403295 6304 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.403327 6301 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.403383 6304 ts_tablet_manager.cc:1434] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.403393 6301 ts_tablet_manager.cc:1403] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251209 03:04:10.403328 6302 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.403448 6304 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.403492 6302 ts_tablet_manager.cc:1403] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.403553 6301 raft_consensus.cc:359] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.403599 6301 raft_consensus.cc:385] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.403626 6301 raft_consensus.cc:740] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.403612 6302 raft_consensus.cc:359] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.403651 6302 raft_consensus.cc:385] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.403669 6302 raft_consensus.cc:740] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.403662 6301 consensus_queue.cc:260] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.403800 6302 consensus_queue.cc:260] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.403843 6301 ts_tablet_manager.cc:1434] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.403867 6304 tablet_bootstrap.cc:654] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.403900 6301 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.404072 6302 ts_tablet_manager.cc:1434] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:04:10.404299 6301 tablet_bootstrap.cc:654] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.404366 6304 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.404407 6304 ts_tablet_manager.cc:1403] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:04:10.404532 6302 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.404927 6302 tablet_bootstrap.cc:654] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.404528 6304 raft_consensus.cc:359] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.405179 6304 raft_consensus.cc:385] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.405212 6304 raft_consensus.cc:740] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.405262 6304 consensus_queue.cc:260] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.405344 6304 ts_tablet_manager.cc:1434] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:04:10.405405 6302 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.405454 6302 ts_tablet_manager.cc:1403] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.405576 6302 raft_consensus.cc:359] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.405658 6302 raft_consensus.cc:385] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.405408 6304 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.405783 6302 raft_consensus.cc:740] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.405875 6302 consensus_queue.cc:260] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.405947 6301 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.405962 6302 ts_tablet_manager.cc:1434] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.406006 6301 ts_tablet_manager.cc:1403] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:04:10.406134 6301 raft_consensus.cc:359] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.406137 6302 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.406208 6301 raft_consensus.cc:385] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.406172 6304 tablet_bootstrap.cc:654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.406239 6301 raft_consensus.cc:740] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.406273 6301 consensus_queue.cc:260] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.406359 6301 ts_tablet_manager.cc:1434] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.406422 6301 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.406638 6302 tablet_bootstrap.cc:654] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.406790 6304 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.406790 6301 tablet_bootstrap.cc:654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.406901 6304 ts_tablet_manager.cc:1403] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.407042 6304 raft_consensus.cc:359] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.407089 6304 raft_consensus.cc:385] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.407106 6304 raft_consensus.cc:740] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.407145 6304 consensus_queue.cc:260] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.407219 6304 ts_tablet_manager.cc:1434] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.407263 6304 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.407371 6301 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.407409 6301 ts_tablet_manager.cc:1403] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.407539 6301 raft_consensus.cc:359] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.407585 6301 raft_consensus.cc:385] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.407599 6301 raft_consensus.cc:740] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.407639 6301 consensus_queue.cc:260] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.407683 6304 tablet_bootstrap.cc:654] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.407717 6301 ts_tablet_manager.cc:1434] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.407764 6301 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.408118 6301 tablet_bootstrap.cc:654] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.408212 6304 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.408258 6304 ts_tablet_manager.cc:1403] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:04:10.408409 6304 raft_consensus.cc:359] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.408470 6304 raft_consensus.cc:385] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.408489 6304 raft_consensus.cc:740] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.408531 6304 consensus_queue.cc:260] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.408613 6304 ts_tablet_manager.cc:1434] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.408624 6301 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.408676 6301 ts_tablet_manager.cc:1403] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.408762 6311 raft_consensus.cc:493] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.408684 6304 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.408821 6311 raft_consensus.cc:515] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.408823 6301 raft_consensus.cc:359] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.408922 6301 raft_consensus.cc:385] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.408941 6301 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.408978 6301 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.408987 6311 leader_election.cc:290] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.409112 6301 ts_tablet_manager.cc:1434] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.409161 6301 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.409318 6302 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.409358 6302 ts_tablet_manager.cc:1403] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20251209 03:04:10.409478 6302 raft_consensus.cc:359] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.409528 6302 raft_consensus.cc:385] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.409539 6301 tablet_bootstrap.cc:654] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.409551 6302 raft_consensus.cc:740] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.409595 6302 consensus_queue.cc:260] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.409709 6302 ts_tablet_manager.cc:1434] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.409163 6304 tablet_bootstrap.cc:654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.410010 6302 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.410365 6302 tablet_bootstrap.cc:654] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.410874 6301 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.410915 6301 ts_tablet_manager.cc:1403] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:04:10.411047 6301 raft_consensus.cc:359] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.411096 6301 raft_consensus.cc:385] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.411116 6301 raft_consensus.cc:740] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.411160 6301 consensus_queue.cc:260] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.411242 6301 ts_tablet_manager.cc:1434] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.411293 6301 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: Bootstrap starting.
I20251209 03:04:10.411665 6301 tablet_bootstrap.cc:654] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.410912 6302 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.412004 6302 ts_tablet_manager.cc:1403] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:04:10.412139 6302 raft_consensus.cc:359] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.412209 6302 raft_consensus.cc:385] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.412237 6302 raft_consensus.cc:740] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.412281 6302 consensus_queue.cc:260] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.412361 6302 ts_tablet_manager.cc:1434] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.412415 6302 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.412781 6302 tablet_bootstrap.cc:654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.413161 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf139ecb17be4ec58d5c23e86eab88a4" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036" is_pre_election: true
I20251209 03:04:10.413244 6255 raft_consensus.cc:2468] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 0.
I20251209 03:04:10.413260 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf139ecb17be4ec58d5c23e86eab88a4" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385" is_pre_election: true
I20251209 03:04:10.412179 6301 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: No bootstrap required, opened a new log
I20251209 03:04:10.413328 6104 raft_consensus.cc:2468] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 0.
I20251209 03:04:10.413362 6301 ts_tablet_manager.cc:1403] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:04:10.413391 6145 leader_election.cc:304] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.413509 6336 raft_consensus.cc:2804] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.413587 6336 raft_consensus.cc:493] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.413621 6336 raft_consensus.cc:3060] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.413755 6302 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.413794 6302 ts_tablet_manager.cc:1403] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.413847 6304 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.413882 6304 ts_tablet_manager.cc:1403] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20251209 03:04:10.413899 6302 raft_consensus.cc:359] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.413931 6302 raft_consensus.cc:385] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.413942 6302 raft_consensus.cc:740] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.413975 6302 consensus_queue.cc:260] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.414012 6304 raft_consensus.cc:359] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.414045 6302 ts_tablet_manager.cc:1434] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.414055 6304 raft_consensus.cc:385] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.414080 6304 raft_consensus.cc:740] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.414115 6304 consensus_queue.cc:260] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.414141 6302 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: Bootstrap starting.
I20251209 03:04:10.414199 6304 ts_tablet_manager.cc:1434] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.414250 6304 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: Bootstrap starting.
I20251209 03:04:10.414403 6336 raft_consensus.cc:515] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.414510 6336 leader_election.cc:290] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.414515 6302 tablet_bootstrap.cc:654] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.414608 6304 tablet_bootstrap.cc:654] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:10.414690 6306 raft_consensus.cc:493] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.414731 6306 raft_consensus.cc:515] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.414613 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf139ecb17be4ec58d5c23e86eab88a4" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:10.414834 6255 raft_consensus.cc:3060] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.414888 6306 leader_election.cc:290] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.413508 6301 raft_consensus.cc:359] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.414988 6301 raft_consensus.cc:385] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.415012 6301 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.415050 6301 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.415151 6301 ts_tablet_manager.cc:1434] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251209 03:04:10.415498 6255 raft_consensus.cc:2468] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 1.
I20251209 03:04:10.415717 6145 leader_election.cc:304] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.415993 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf139ecb17be4ec58d5c23e86eab88a4" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:10.416055 6104 raft_consensus.cc:3060] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.416703 6104 raft_consensus.cc:2468] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 1.
I20251209 03:04:10.416927 6336 raft_consensus.cc:2804] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.416975 6336 raft_consensus.cc:697] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:10.417009 6336 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.417443 6000 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.418027 6302 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: No bootstrap required, opened a new log
I20251209 03:04:10.418031 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" is_pre_election: true
I20251209 03:04:10.418104 6180 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 0.
I20251209 03:04:10.418104 6302 ts_tablet_manager.cc:1403] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20251209 03:04:10.418244 6069 leader_election.cc:304] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.418258 6302 raft_consensus.cc:359] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.418329 6302 raft_consensus.cc:385] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.418357 6302 raft_consensus.cc:740] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.418396 6302 consensus_queue.cc:260] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.418483 6302 ts_tablet_manager.cc:1434] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.418510 6304 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: No bootstrap required, opened a new log
I20251209 03:04:10.418560 6304 ts_tablet_manager.cc:1403] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251209 03:04:10.418411 6306 raft_consensus.cc:2804] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.418684 6306 raft_consensus.cc:493] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.418735 6306 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.419005 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036" is_pre_election: true
I20251209 03:04:10.419070 6255 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 0.
I20251209 03:04:10.419286 6306 raft_consensus.cc:515] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.419384 6306 leader_election.cc:290] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.418696 6304 raft_consensus.cc:359] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.419482 6304 raft_consensus.cc:385] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:10.419502 6304 raft_consensus.cc:740] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Initialized, Role: FOLLOWER
I20251209 03:04:10.419545 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:10.419543 6304 consensus_queue.cc:260] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.419549 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:10.419649 6255 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.419657 6180 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.419687 6304 ts_tablet_manager.cc:1434] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:04:10.420202 6255 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 1.
I20251209 03:04:10.420365 6069 leader_election.cc:304] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.420461 6306 raft_consensus.cc:2804] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.420511 6306 raft_consensus.cc:697] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:10.420528 6180 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 1.
I20251209 03:04:10.420557 6306 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.421113 6000 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 0 to 1, leader changed from <none> to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.456460 6306 raft_consensus.cc:493] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.456547 6306 raft_consensus.cc:515] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.456667 6306 leader_election.cc:290] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.456873 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036" is_pre_election: true
I20251209 03:04:10.456868 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" is_pre_election: true
I20251209 03:04:10.456954 6255 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 0.
I20251209 03:04:10.456998 6180 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 0.
I20251209 03:04:10.457110 6069 leader_election.cc:304] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.457230 6306 raft_consensus.cc:2804] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.457266 6306 raft_consensus.cc:493] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.457285 6306 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.457914 6306 raft_consensus.cc:515] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.458024 6306 leader_election.cc:290] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.458161 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:10.458199 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:10.458214 6180 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.458276 6255 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.458787 6180 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 1.
I20251209 03:04:10.458793 6255 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 1.
I20251209 03:04:10.458945 6069 leader_election.cc:304] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.459025 6306 raft_consensus.cc:2804] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.459076 6306 raft_consensus.cc:697] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:10.459122 6306 consensus_queue.cc:237] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.459697 6000 catalog_manager.cc:5654] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 0 to 1, leader changed from <none> to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.460800 6306 raft_consensus.cc:493] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.460845 6306 raft_consensus.cc:515] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.460934 6306 leader_election.cc:290] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.461097 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ead69a2942346dca25124b075136d12" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036" is_pre_election: true
I20251209 03:04:10.461145 6255 raft_consensus.cc:2468] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 0.
I20251209 03:04:10.461120 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ead69a2942346dca25124b075136d12" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" is_pre_election: true
I20251209 03:04:10.461216 6180 raft_consensus.cc:2468] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 0.
I20251209 03:04:10.461328 6069 leader_election.cc:304] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.461436 6306 raft_consensus.cc:2804] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.461477 6306 raft_consensus.cc:493] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.461490 6306 raft_consensus.cc:3060] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.462002 6306 raft_consensus.cc:515] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.462092 6306 leader_election.cc:290] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.462217 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ead69a2942346dca25124b075136d12" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:10.462222 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ead69a2942346dca25124b075136d12" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:10.462296 6180 raft_consensus.cc:3060] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.462303 6255 raft_consensus.cc:3060] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.462847 6180 raft_consensus.cc:2468] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 1.
I20251209 03:04:10.462847 6255 raft_consensus.cc:2468] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 1.
I20251209 03:04:10.463011 6069 leader_election.cc:304] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.463081 6306 raft_consensus.cc:2804] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.463147 6306 raft_consensus.cc:697] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:10.463196 6306 consensus_queue.cc:237] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.463689 6000 catalog_manager.cc:5654] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 0 to 1, leader changed from <none> to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.471063 6313 raft_consensus.cc:493] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.471128 6313 raft_consensus.cc:515] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.471279 6313 leader_election.cc:290] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:10.473865 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2beec1d9e1ff42cea8ff5a8e9024a6fe" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" is_pre_election: true
I20251209 03:04:10.473868 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2beec1d9e1ff42cea8ff5a8e9024a6fe" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385" is_pre_election: true
I20251209 03:04:10.473991 6180 raft_consensus.cc:2468] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 0.
I20251209 03:04:10.473995 6104 raft_consensus.cc:2468] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 0.
I20251209 03:04:10.474138 6220 leader_election.cc:304] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.474242 6313 raft_consensus.cc:2804] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.474292 6313 raft_consensus.cc:493] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.474335 6313 raft_consensus.cc:3060] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.474915 6313 raft_consensus.cc:515] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.475025 6313 leader_election.cc:290] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:10.475165 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2beec1d9e1ff42cea8ff5a8e9024a6fe" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:10.475167 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2beec1d9e1ff42cea8ff5a8e9024a6fe" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:10.475256 6104 raft_consensus.cc:3060] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.475262 6180 raft_consensus.cc:3060] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.475840 6104 raft_consensus.cc:2468] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 1.
I20251209 03:04:10.475840 6180 raft_consensus.cc:2468] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 1.
I20251209 03:04:10.476023 6220 leader_election.cc:304] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.476099 6313 raft_consensus.cc:2804] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.476235 6313 raft_consensus.cc:697] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:10.476284 6313 consensus_queue.cc:237] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.476827 6000 catalog_manager.cc:5654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } }
I20251209 03:04:10.488435 6306 raft_consensus.cc:493] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.488505 6306 raft_consensus.cc:515] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.488621 6306 leader_election.cc:290] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.488770 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e6b3f62d7894c69b4cf99bf1ad942ce" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" is_pre_election: true
I20251209 03:04:10.488778 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e6b3f62d7894c69b4cf99bf1ad942ce" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036" is_pre_election: true
I20251209 03:04:10.488866 6180 raft_consensus.cc:2468] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 0.
I20251209 03:04:10.488873 6255 raft_consensus.cc:2468] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 0.
I20251209 03:04:10.489007 6069 leader_election.cc:304] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.489097 6306 raft_consensus.cc:2804] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.489140 6306 raft_consensus.cc:493] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.489159 6306 raft_consensus.cc:3060] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.489737 6306 raft_consensus.cc:515] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.489835 6306 leader_election.cc:290] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.489955 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e6b3f62d7894c69b4cf99bf1ad942ce" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:10.489964 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e6b3f62d7894c69b4cf99bf1ad942ce" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:10.490034 6180 raft_consensus.cc:3060] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.490041 6255 raft_consensus.cc:3060] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.490581 6180 raft_consensus.cc:2468] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 1.
I20251209 03:04:10.490581 6255 raft_consensus.cc:2468] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 1.
I20251209 03:04:10.490754 6069 leader_election.cc:304] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.490815 6306 raft_consensus.cc:2804] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.490875 6306 raft_consensus.cc:697] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:10.490921 6306 consensus_queue.cc:237] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.491464 6000 catalog_manager.cc:5654] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 0 to 1, leader changed from <none> to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.509907 6313 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.509986 6313 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.510097 6313 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:10.510298 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" is_pre_election: true
I20251209 03:04:10.510301 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385" is_pre_election: true
I20251209 03:04:10.510387 6180 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 0.
I20251209 03:04:10.510412 6104 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 0.
I20251209 03:04:10.510517 6220 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.510605 6313 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.510653 6313 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.510666 6313 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.511209 6313 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.511305 6313 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:10.511462 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:10.511482 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:10.511512 6104 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.511523 6180 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.511960 6180 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 1.
I20251209 03:04:10.511965 6104 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 1.
I20251209 03:04:10.512115 6220 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.512192 6313 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.512236 6313 raft_consensus.cc:697] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:10.512274 6313 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:10.512794 6000 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.559636 6311 raft_consensus.cc:493] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.559736 6311 raft_consensus.cc:515] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.559864 6311 leader_election.cc:290] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.560051 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9a5e7d1a2de4913a09dfeb084d79ae1" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385" is_pre_election: true
I20251209 03:04:10.560086 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9a5e7d1a2de4913a09dfeb084d79ae1" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036" is_pre_election: true
I20251209 03:04:10.560158 6104 raft_consensus.cc:2468] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 0.
I20251209 03:04:10.560163 6255 raft_consensus.cc:2468] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 0.
I20251209 03:04:10.560338 6145 leader_election.cc:304] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.560408 6311 raft_consensus.cc:2804] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.560456 6311 raft_consensus.cc:493] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.560482 6311 raft_consensus.cc:3060] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.561102 6311 raft_consensus.cc:515] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.561208 6311 leader_election.cc:290] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.561332 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9a5e7d1a2de4913a09dfeb084d79ae1" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:10.561403 6104 raft_consensus.cc:3060] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.561430 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9a5e7d1a2de4913a09dfeb084d79ae1" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:10.561478 6255 raft_consensus.cc:3060] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.562042 6255 raft_consensus.cc:2468] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 1.
I20251209 03:04:10.562042 6104 raft_consensus.cc:2468] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 1.
I20251209 03:04:10.562227 6145 leader_election.cc:304] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.562304 6311 raft_consensus.cc:2804] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.562366 6311 raft_consensus.cc:697] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:10.562420 6311 consensus_queue.cc:237] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.563010 6000 catalog_manager.cc:5654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.575594 6313 raft_consensus.cc:493] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.575683 6313 raft_consensus.cc:515] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.575790 6313 leader_election.cc:290] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:10.575986 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385" is_pre_election: true
I20251209 03:04:10.575986 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" is_pre_election: true
I20251209 03:04:10.576097 6104 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 0.
I20251209 03:04:10.576100 6180 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 0.
I20251209 03:04:10.576241 6220 leader_election.cc:304] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.576323 6313 raft_consensus.cc:2804] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.576372 6313 raft_consensus.cc:493] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.576391 6313 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.576911 6313 raft_consensus.cc:515] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.577015 6313 leader_election.cc:290] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:10.577153 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:10.577180 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:10.577204 6104 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.577248 6180 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.577704 6104 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 1.
I20251209 03:04:10.577841 6220 leader_election.cc:304] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.577910 6313 raft_consensus.cc:2804] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.577975 6180 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 1.
I20251209 03:04:10.577982 6313 raft_consensus.cc:697] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:10.578048 6313 consensus_queue.cc:237] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.578727 6000 catalog_manager.cc:5654] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } }
I20251209 03:04:10.591034 6311 raft_consensus.cc:493] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.591112 6311 raft_consensus.cc:515] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.591219 6311 leader_election.cc:290] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.591398 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f71e8ee2ce1242608207622b267d96ae" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385" is_pre_election: true
I20251209 03:04:10.591432 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f71e8ee2ce1242608207622b267d96ae" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036" is_pre_election: true
I20251209 03:04:10.591471 6104 raft_consensus.cc:2468] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 0.
I20251209 03:04:10.591486 6255 raft_consensus.cc:2468] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 0.
I20251209 03:04:10.591593 6145 leader_election.cc:304] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.591675 6311 raft_consensus.cc:2804] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.591720 6311 raft_consensus.cc:493] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.591745 6311 raft_consensus.cc:3060] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.592299 6311 raft_consensus.cc:515] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.592398 6311 leader_election.cc:290] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.592522 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f71e8ee2ce1242608207622b267d96ae" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:10.592533 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f71e8ee2ce1242608207622b267d96ae" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:10.592579 6255 raft_consensus.cc:3060] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.592594 6104 raft_consensus.cc:3060] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.593019 6255 raft_consensus.cc:2468] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 1.
I20251209 03:04:10.593039 6104 raft_consensus.cc:2468] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 1.
I20251209 03:04:10.593168 6145 leader_election.cc:304] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.593254 6311 raft_consensus.cc:2804] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.593310 6311 raft_consensus.cc:697] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:10.593357 6311 consensus_queue.cc:237] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.593919 6000 catalog_manager.cc:5654] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.596756 6311 raft_consensus.cc:493] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.596808 6311 raft_consensus.cc:515] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.596889 6311 leader_election.cc:290] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.597023 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385" is_pre_election: true
I20251209 03:04:10.597029 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036" is_pre_election: true
I20251209 03:04:10.597091 6104 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 0.
I20251209 03:04:10.597097 6255 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 0.
I20251209 03:04:10.597211 6145 leader_election.cc:304] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.597291 6311 raft_consensus.cc:2804] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.597329 6311 raft_consensus.cc:493] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.597352 6311 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.597874 6311 raft_consensus.cc:515] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.597970 6311 leader_election.cc:290] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:10.598111 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:10.598110 6255 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:10.598191 6104 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.598197 6255 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.598696 6255 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 1.
I20251209 03:04:10.598704 6104 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 1.
I20251209 03:04:10.598857 6145 leader_election.cc:304] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:10.598935 6311 raft_consensus.cc:2804] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.598984 6311 raft_consensus.cc:697] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:10.599038 6311 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.599587 6000 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:10.766918 6313 raft_consensus.cc:493] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:10.767045 6313 raft_consensus.cc:515] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.767210 6313 leader_election.cc:290] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:10.767531 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "439ea3fec68444aeb8cc383b977a7f48" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" is_pre_election: true
I20251209 03:04:10.767554 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "439ea3fec68444aeb8cc383b977a7f48" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385" is_pre_election: true
I20251209 03:04:10.767639 6180 raft_consensus.cc:2468] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 0.
I20251209 03:04:10.767656 6104 raft_consensus.cc:2468] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 0.
I20251209 03:04:10.767779 6220 leader_election.cc:304] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.768345 6313 raft_consensus.cc:2804] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:10.768409 6313 raft_consensus.cc:493] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:10.768432 6313 raft_consensus.cc:3060] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.769009 6313 raft_consensus.cc:515] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.769114 6313 leader_election.cc:290] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:10.769263 6104 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "439ea3fec68444aeb8cc383b977a7f48" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:10.769287 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "439ea3fec68444aeb8cc383b977a7f48" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:10.769325 6104 raft_consensus.cc:3060] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.769331 6180 raft_consensus.cc:3060] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:10.769853 6180 raft_consensus.cc:2468] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 1.
I20251209 03:04:10.769917 6104 raft_consensus.cc:2468] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 1.
I20251209 03:04:10.770017 6220 leader_election.cc:304] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:10.770118 6313 raft_consensus.cc:2804] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:10.770164 6313 raft_consensus.cc:697] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:10.770210 6313 consensus_queue.cc:237] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:10.770766 6000 catalog_manager.cc:5654] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } }
I20251209 03:04:10.817797 6104 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818116 6311 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.818342 6104 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818362 6180 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818427 6178 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818519 6255 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818552 6103 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818607 6254 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818586 6311 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.818653 6102 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818717 6253 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818747 6101 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818825 6100 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818845 6252 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818918 6099 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.818384 6179 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.819391 6385 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.819448 6311 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.819510 6385 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.819541 6311 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.819584 6355 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.819821 6355 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.819905 6355 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.819970 6355 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.820019 6355 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.820065 6355 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.821220 6179 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.821300 6180 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.821451 6177 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.821527 6178 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.822686 6344 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.822809 6306 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.822837 6344 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.822909 6306 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.819653 6311 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20251209 03:04:10.823065 6252 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.823148 6255 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.822959 6251 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.822918 6254 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.819582 6385 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20251209 03:04:10.826120 6344 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.826218 6344 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.826282 6344 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.826339 6306 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.826103 6377 mvcc.cc:204] Tried to move back new op lower bound from 7230461750547091456 to 7230461749151424512. Current Snapshot: MvccSnapshot[applied={T|T < 7230461750547091456}]
I20251209 03:04:10.827003 6102 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.827805 6386 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.828249 6175 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:10.829352 6313 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:10.830643 6380 mvcc.cc:204] Tried to move back new op lower bound from 7230461750547091456 to 7230461749151424512. Current Snapshot: MvccSnapshot[applied={T|T < 7230461750547091456}]
I20251209 03:04:10.833303 6402 mvcc.cc:204] Tried to move back new op lower bound from 7230461750552612864 to 7230461749630902272. Current Snapshot: MvccSnapshot[applied={T|T < 7230461750548729856}]
I20251209 03:04:10.920994 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:10.923002 6453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:10.923226 6451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:10.923296 6450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:10.923750 2731 server_base.cc:1047] running on GCE node
I20251209 03:04:10.923877 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:10.923911 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:10.923928 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249450923929 us; error 0 us; skew 500 ppm
I20251209 03:04:10.924686 2731 webserver.cc:492] Webserver started at http://127.2.170.196:38549/ using document root <none> and password file <none>
I20251209 03:04:10.924759 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:10.924808 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:10.924851 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:10.925094 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-3-root/instance:
uuid: "37b7f916c7c843dfa479e106f762cfff"
format_stamp: "Formatted at 2025-12-09 03:04:10 on dist-test-slave-4cb0"
I20251209 03:04:10.926040 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.926509 6458 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.926661 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.926705 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-3-root
uuid: "37b7f916c7c843dfa479e106f762cfff"
format_stamp: "Formatted at 2025-12-09 03:04:10 on dist-test-slave-4cb0"
I20251209 03:04:10.926748 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:10.936607 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:10.936792 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:10.936906 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:10.937079 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:10.937106 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.937134 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:10.937157 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.940394 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.196:43979
I20251209 03:04:10.940446 6521 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.196:43979 every 8 connection(s)
I20251209 03:04:10.940773 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:10.941947 6526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:10.942045 6527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:10.942096 2731 server_base.cc:1047] running on GCE node
W20251209 03:04:10.942183 6529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:10.942353 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:10.942392 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:10.942410 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249450942410 us; error 0 us; skew 500 ppm
I20251209 03:04:10.943097 2731 webserver.cc:492] Webserver started at http://127.2.170.197:33009/ using document root <none> and password file <none>
I20251209 03:04:10.943185 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:10.943224 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:10.943269 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:10.943516 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-4-root/instance:
uuid: "2551ae093c914660b7f6e42bbea920fa"
format_stamp: "Formatted at 2025-12-09 03:04:10 on dist-test-slave-4cb0"
I20251209 03:04:10.944205 6522 heartbeater.cc:344] Connected to a master server at 127.2.170.254:39481
I20251209 03:04:10.944264 6522 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:10.944411 6522 heartbeater.cc:507] Master 127.2.170.254:39481 requested a full tablet report, sending...
I20251209 03:04:10.944588 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:10.944669 5999 ts_manager.cc:194] Registered new tserver with Master: 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:10.945075 5999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53176
I20251209 03:04:10.945142 6534 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.945231 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20251209 03:04:10.945269 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-4-root
uuid: "2551ae093c914660b7f6e42bbea920fa"
format_stamp: "Formatted at 2025-12-09 03:04:10 on dist-test-slave-4cb0"
I20251209 03:04:10.945312 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-4-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-4-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-4-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:10.949482 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:10.949712 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:10.949815 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:10.949980 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:10.950004 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.950022 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:10.950047 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.953118 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.197:41607
I20251209 03:04:10.953241 6597 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.197:41607 every 8 connection(s)
I20251209 03:04:10.953518 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:10.954690 6601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:10.954870 2731 server_base.cc:1047] running on GCE node
W20251209 03:04:10.954906 6604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:10.955124 6602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:10.955312 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:10.955358 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:10.955374 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249450955375 us; error 0 us; skew 500 ppm
I20251209 03:04:10.956152 2731 webserver.cc:492] Webserver started at http://127.2.170.198:46021/ using document root <none> and password file <none>
I20251209 03:04:10.956231 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:10.956274 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:10.956323 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:10.956593 6598 heartbeater.cc:344] Connected to a master server at 127.2.170.254:39481
I20251209 03:04:10.956595 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-5-root/instance:
uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
format_stamp: "Formatted at 2025-12-09 03:04:10 on dist-test-slave-4cb0"
I20251209 03:04:10.956665 6598 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:10.956769 6598 heartbeater.cc:507] Master 127.2.170.254:39481 requested a full tablet report, sending...
I20251209 03:04:10.956980 5999 ts_manager.cc:194] Registered new tserver with Master: 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607)
I20251209 03:04:10.957315 5999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53192
I20251209 03:04:10.957712 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:10.958165 6609 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.958292 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.958334 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-5-root
uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
format_stamp: "Formatted at 2025-12-09 03:04:10 on dist-test-slave-4cb0"
I20251209 03:04:10.958376 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-5-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-5-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-5-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:10.968633 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:10.968869 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:10.969009 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:10.969173 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:10.969200 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.969228 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:10.969244 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:10.972699 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.198:34151
I20251209 03:04:10.972827 6672 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.198:34151 every 8 connection(s)
I20251209 03:04:10.975818 6673 heartbeater.cc:344] Connected to a master server at 127.2.170.254:39481
I20251209 03:04:10.975878 6673 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:10.975970 6673 heartbeater.cc:507] Master 127.2.170.254:39481 requested a full tablet report, sending...
I20251209 03:04:10.976161 5999 ts_manager.cc:194] Registered new tserver with Master: 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151)
I20251209 03:04:10.976488 5999 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53196
I20251209 03:04:11.235695 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:11.238876 6250 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:11.238996 6250 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Received request to transfer leadership to TS d36845214167428eb93883ecbe8f5385
I20251209 03:04:11.239162 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
W20251209 03:04:11.239208 6049 auto_rebalancer.cc:253] failed to send replica move request: Remote error: Service unavailable: leader transfer in progress
I20251209 03:04:11.239238 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251209 03:04:11.239607 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 5ef8ec5789d64b8f931598cfcded95f2, TS a6fcaebe8f1f4a319f00f9666d8ba592 -> TS 03aecf9a33ec4b4ba130772d3c35d33e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
W20251209 03:04:11.239650 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 5ef8ec5789d64b8f931598cfcded95f2, TS a6fcaebe8f1f4a319f00f9666d8ba592 -> TS 03aecf9a33ec4b4ba130772d3c35d33e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
W20251209 03:04:11.239972 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet cf139ecb17be4ec58d5c23e86eab88a4, TS b2fc3e20641e45a4b62732385fd4e036 -> TS 37b7f916c7c843dfa479e106f762cfff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.240001 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet cf139ecb17be4ec58d5c23e86eab88a4, TS b2fc3e20641e45a4b62732385fd4e036 -> TS 37b7f916c7c843dfa479e106f762cfff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.240268 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet a9a5e7d1a2de4913a09dfeb084d79ae1, TS d36845214167428eb93883ecbe8f5385 -> TS 2551ae093c914660b7f6e42bbea920fa move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.240295 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet a9a5e7d1a2de4913a09dfeb084d79ae1, TS d36845214167428eb93883ecbe8f5385 -> TS 2551ae093c914660b7f6e42bbea920fa move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.240654 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 0e6b3f62d7894c69b4cf99bf1ad942ce, TS b2fc3e20641e45a4b62732385fd4e036 -> TS 2551ae093c914660b7f6e42bbea920fa move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.240687 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 0e6b3f62d7894c69b4cf99bf1ad942ce, TS b2fc3e20641e45a4b62732385fd4e036 -> TS 2551ae093c914660b7f6e42bbea920fa move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.241075 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 7ead69a2942346dca25124b075136d12, TS a6fcaebe8f1f4a319f00f9666d8ba592 -> TS 03aecf9a33ec4b4ba130772d3c35d33e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.241112 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 7ead69a2942346dca25124b075136d12, TS a6fcaebe8f1f4a319f00f9666d8ba592 -> TS 03aecf9a33ec4b4ba130772d3c35d33e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.241513 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet f9a52de6742a4e508e06374063561398, TS d36845214167428eb93883ecbe8f5385 -> TS 37b7f916c7c843dfa479e106f762cfff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:11.241552 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet f9a52de6742a4e508e06374063561398, TS d36845214167428eb93883ecbe8f5385 -> TS 37b7f916c7c843dfa479e106f762cfff move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:11.469322 6679 raft_consensus.cc:993] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:11.469429 6449 raft_consensus.cc:1081] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Signalling peer d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:11.469698 6100 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:58884
I20251209 03:04:11.469797 6100 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:11.469820 6100 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:11.470431 6100 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:11.470546 6100 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 2 election: Requested vote from peers b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:11.470736 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:11.470736 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:11.470827 6252 raft_consensus.cc:3055] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:11.470850 6180 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:11.470850 6252 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:11.471043 6252 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:11.471124 6252 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:11.471603 6180 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 2.
I20251209 03:04:11.471756 6252 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 2.
I20251209 03:04:11.471776 6069 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:11.471863 6439 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:11.471952 6439 raft_consensus.cc:697] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 2 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:11.472010 6439 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:11.472481 5999 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 1 to 2, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 2 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:11.926918 6180 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251209 03:04:11.927145 6439 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:11.928563 6252 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251209 03:04:11.928766 6686 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:11.945710 6522 heartbeater.cc:499] Master 127.2.170.254:39481 was elected leader, sending a full tablet report...
I20251209 03:04:11.957952 6598 heartbeater.cc:499] Master 127.2.170.254:39481 was elected leader, sending a full tablet report...
I20251209 03:04:11.976938 6673 heartbeater.cc:499] Master 127.2.170.254:39481 was elected leader, sending a full tablet report...
I20251209 03:04:12.242549 6252 consensus_queue.cc:237] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:12.243206 6180 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251209 03:04:12.243216 6100 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251209 03:04:12.243443 6449 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251209 03:04:12.243631 6691 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251209 03:04:12.244366 6691 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:12.244488 6180 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:12.244580 6100 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:12.245625 6001 catalog_manager.cc:5654] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
W20251209 03:04:12.246856 6218 consensus_peers.cc:597] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 -> Peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979): Couldn't send request to peer 37b7f916c7c843dfa479e106f762cfff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 439ea3fec68444aeb8cc383b977a7f48. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:12.248756 6100 consensus_queue.cc:237] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:12.249429 6252 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251209 03:04:12.249444 6180 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251209 03:04:12.249620 6439 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251209 03:04:12.249791 6680 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251209 03:04:12.250285 6686 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:12.250398 6252 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:12.251549 5998 catalog_manager.cc:5654] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index -1 to 29, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:12.251716 6180 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
W20251209 03:04:12.252480 6070 consensus_peers.cc:597] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 -> Peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607): Couldn't send request to peer 2551ae093c914660b7f6e42bbea920fa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ead69a2942346dca25124b075136d12. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:12.254786 6180 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:12.255296 6100 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:12.255389 6252 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:12.255551 6709 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:12.255687 6710 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:12.256430 6384 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:12.256491 6100 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:12.257524 6001 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:12.257555 6252 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:12.257709 6250 consensus_queue.cc:237] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:12.258188 6100 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:12.258491 6180 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
W20251209 03:04:12.258663 6146 consensus_peers.cc:597] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151): Couldn't send request to peer 03aecf9a33ec4b4ba130772d3c35d33e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf139ecb17be4ec58d5c23e86eab88a4. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:12.258499 6692 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:12.258877 6449 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:12.259563 6691 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:12.259668 6100 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:12.259696 6180 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:12.260751 5998 catalog_manager.cc:5654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index -1 to 30, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
W20251209 03:04:12.261418 6221 consensus_peers.cc:597] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 -> Peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607): Couldn't send request to peer 2551ae093c914660b7f6e42bbea920fa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2beec1d9e1ff42cea8ff5a8e9024a6fe. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:12.261583 6180 consensus_queue.cc:237] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:12.262094 6100 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:12.262108 6252 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
W20251209 03:04:12.262190 6146 consensus_peers.cc:597] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151): Couldn't send request to peer 03aecf9a33ec4b4ba130772d3c35d33e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f71e8ee2ce1242608207622b267d96ae. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:12.262290 6384 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:12.262382 6710 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:12.263020 6709 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:12.263123 6100 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:12.263139 6252 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:12.264166 5999 catalog_manager.cc:5654] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:12.264163 6180 consensus_queue.cc:237] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 1.28, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:12.264780 6100 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251209 03:04:12.264780 6252 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 28. Preceding OpId from leader: term: 1 index: 29. (index mismatch)
I20251209 03:04:12.265098 6384 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251209 03:04:12.265230 6709 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251209 03:04:12.265856 6384 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:12.265965 6252 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:12.266909 6001 catalog_manager.cc:5654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:12.267258 6100 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
W20251209 03:04:12.267421 6143 consensus_peers.cc:597] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979): Couldn't send request to peer 37b7f916c7c843dfa479e106f762cfff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9a5e7d1a2de4913a09dfeb084d79ae1. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:12.655666 6724 ts_tablet_manager.cc:933] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: Initiating tablet copy from peer b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:12.655885 6724 tablet_copy_client.cc:323] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.195:34597
I20251209 03:04:12.658610 6265 tablet_copy_service.cc:140] P b2fc3e20641e45a4b62732385fd4e036: Received BeginTabletCopySession request for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe from peer 2551ae093c914660b7f6e42bbea920fa ({username='slave'} at 127.0.0.1:35118)
I20251209 03:04:12.658677 6265 tablet_copy_service.cc:161] P b2fc3e20641e45a4b62732385fd4e036: Beginning new tablet copy session on tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe from peer 2551ae093c914660b7f6e42bbea920fa at {username='slave'} at 127.0.0.1:35118: session id = 2551ae093c914660b7f6e42bbea920fa-2beec1d9e1ff42cea8ff5a8e9024a6fe
I20251209 03:04:12.659116 6265 tablet_copy_source_session.cc:215] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:12.659443 6724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:12.661085 6724 tablet_copy_client.cc:806] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:12.661237 6724 tablet_copy_client.cc:670] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:12.661760 6725 tablet_copy_client.cc:1025] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:12.767107 6728 ts_tablet_manager.cc:933] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: Initiating tablet copy from peer b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:12.767331 6728 tablet_copy_client.cc:323] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.195:34597
I20251209 03:04:12.770255 6265 tablet_copy_service.cc:140] P b2fc3e20641e45a4b62732385fd4e036: Received BeginTabletCopySession request for tablet 439ea3fec68444aeb8cc383b977a7f48 from peer 37b7f916c7c843dfa479e106f762cfff ({username='slave'} at 127.0.0.1:35124)
I20251209 03:04:12.770334 6265 tablet_copy_service.cc:161] P b2fc3e20641e45a4b62732385fd4e036: Beginning new tablet copy session on tablet 439ea3fec68444aeb8cc383b977a7f48 from peer 37b7f916c7c843dfa479e106f762cfff at {username='slave'} at 127.0.0.1:35124: session id = 37b7f916c7c843dfa479e106f762cfff-439ea3fec68444aeb8cc383b977a7f48
I20251209 03:04:12.770859 6265 tablet_copy_source_session.cc:215] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:12.771200 6728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 439ea3fec68444aeb8cc383b977a7f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:12.772753 6728 tablet_copy_client.cc:806] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:12.772871 6728 tablet_copy_client.cc:670] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:12.773407 6729 tablet_copy_client.cc:1025] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:12.801383 6731 ts_tablet_manager.cc:933] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: Initiating tablet copy from peer a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:12.801595 6731 tablet_copy_client.cc:323] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42307
I20251209 03:04:12.804498 6190 tablet_copy_service.cc:140] P a6fcaebe8f1f4a319f00f9666d8ba592: Received BeginTabletCopySession request for tablet cf139ecb17be4ec58d5c23e86eab88a4 from peer 03aecf9a33ec4b4ba130772d3c35d33e ({username='slave'} at 127.0.0.1:43178)
I20251209 03:04:12.804566 6190 tablet_copy_service.cc:161] P a6fcaebe8f1f4a319f00f9666d8ba592: Beginning new tablet copy session on tablet cf139ecb17be4ec58d5c23e86eab88a4 from peer 03aecf9a33ec4b4ba130772d3c35d33e at {username='slave'} at 127.0.0.1:43178: session id = 03aecf9a33ec4b4ba130772d3c35d33e-cf139ecb17be4ec58d5c23e86eab88a4
I20251209 03:04:12.805050 6190 tablet_copy_source_session.cc:215] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:12.805419 6731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf139ecb17be4ec58d5c23e86eab88a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:12.806784 6731 tablet_copy_client.cc:806] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:12.806898 6731 tablet_copy_client.cc:670] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:12.807317 6732 tablet_copy_client.cc:1025] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:12.872469 6735 ts_tablet_manager.cc:933] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: Initiating tablet copy from peer d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899)
I20251209 03:04:12.872695 6735 tablet_copy_client.cc:323] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.193:43899
I20251209 03:04:12.875679 6114 tablet_copy_service.cc:140] P d36845214167428eb93883ecbe8f5385: Received BeginTabletCopySession request for tablet 7ead69a2942346dca25124b075136d12 from peer 2551ae093c914660b7f6e42bbea920fa ({username='slave'} at 127.0.0.1:58920)
I20251209 03:04:12.875761 6114 tablet_copy_service.cc:161] P d36845214167428eb93883ecbe8f5385: Beginning new tablet copy session on tablet 7ead69a2942346dca25124b075136d12 from peer 2551ae093c914660b7f6e42bbea920fa at {username='slave'} at 127.0.0.1:58920: session id = 2551ae093c914660b7f6e42bbea920fa-7ead69a2942346dca25124b075136d12
I20251209 03:04:12.876277 6114 tablet_copy_source_session.cc:215] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:12.876595 6735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ead69a2942346dca25124b075136d12. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:12.877951 6735 tablet_copy_client.cc:806] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:12.878069 6735 tablet_copy_client.cc:670] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:12.878563 6736 tablet_copy_client.cc:1025] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:12.879688 6738 ts_tablet_manager.cc:933] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: Initiating tablet copy from peer a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:12.879871 6738 tablet_copy_client.cc:323] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42307
I20251209 03:04:12.880065 6190 tablet_copy_service.cc:140] P a6fcaebe8f1f4a319f00f9666d8ba592: Received BeginTabletCopySession request for tablet f71e8ee2ce1242608207622b267d96ae from peer 03aecf9a33ec4b4ba130772d3c35d33e ({username='slave'} at 127.0.0.1:43178)
I20251209 03:04:12.880120 6190 tablet_copy_service.cc:161] P a6fcaebe8f1f4a319f00f9666d8ba592: Beginning new tablet copy session on tablet f71e8ee2ce1242608207622b267d96ae from peer 03aecf9a33ec4b4ba130772d3c35d33e at {username='slave'} at 127.0.0.1:43178: session id = 03aecf9a33ec4b4ba130772d3c35d33e-f71e8ee2ce1242608207622b267d96ae
I20251209 03:04:12.880645 6190 tablet_copy_source_session.cc:215] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:12.880911 6738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71e8ee2ce1242608207622b267d96ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:12.881990 6738 tablet_copy_client.cc:806] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:12.882098 6738 tablet_copy_client.cc:670] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:12.882556 6739 tablet_copy_client.cc:1025] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:12.884403 6740 ts_tablet_manager.cc:933] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: Initiating tablet copy from peer a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:12.884605 6740 tablet_copy_client.cc:323] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42307
I20251209 03:04:12.887574 6190 tablet_copy_service.cc:140] P a6fcaebe8f1f4a319f00f9666d8ba592: Received BeginTabletCopySession request for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 from peer 37b7f916c7c843dfa479e106f762cfff ({username='slave'} at 127.0.0.1:43190)
I20251209 03:04:12.887636 6190 tablet_copy_service.cc:161] P a6fcaebe8f1f4a319f00f9666d8ba592: Beginning new tablet copy session on tablet a9a5e7d1a2de4913a09dfeb084d79ae1 from peer 37b7f916c7c843dfa479e106f762cfff at {username='slave'} at 127.0.0.1:43190: session id = 37b7f916c7c843dfa479e106f762cfff-a9a5e7d1a2de4913a09dfeb084d79ae1
I20251209 03:04:12.888130 6190 tablet_copy_source_session.cc:215] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:12.888459 6740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a5e7d1a2de4913a09dfeb084d79ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:12.889724 6740 tablet_copy_client.cc:806] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:12.889833 6740 tablet_copy_client.cc:670] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:12.890280 6741 tablet_copy_client.cc:1025] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:13.239377 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:13.239602 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:13.239666 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:14.662021 6724 tablet_copy_client.cc:538] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:14.663183 6724 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: Bootstrap starting.
I20251209 03:04:14.667819 6724 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=124 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:14.668020 6724 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: Bootstrap complete.
I20251209 03:04:14.668128 6724 ts_tablet_manager.cc:1403] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251209 03:04:14.668301 6724 raft_consensus.cc:359] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:14.668370 6724 raft_consensus.cc:740] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2551ae093c914660b7f6e42bbea920fa, State: Initialized, Role: LEARNER
I20251209 03:04:14.668398 6724 consensus_queue.cc:260] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:14.668546 6724 ts_tablet_manager.cc:1434] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: Time spent starting tablet: real 0.000s user 0.003s sys 0.000s
I20251209 03:04:14.668718 6265 tablet_copy_service.cc:342] P b2fc3e20641e45a4b62732385fd4e036: Request end of tablet copy session 2551ae093c914660b7f6e42bbea920fa-2beec1d9e1ff42cea8ff5a8e9024a6fe received from {username='slave'} at 127.0.0.1:35118
I20251209 03:04:14.668771 6265 tablet_copy_service.cc:434] P b2fc3e20641e45a4b62732385fd4e036: ending tablet copy session 2551ae093c914660b7f6e42bbea920fa-2beec1d9e1ff42cea8ff5a8e9024a6fe on tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe with peer 2551ae093c914660b7f6e42bbea920fa
I20251209 03:04:14.773622 6728 tablet_copy_client.cc:538] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:14.774618 6728 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap starting.
I20251209 03:04:14.779318 6728 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:14.779562 6728 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap complete.
I20251209 03:04:14.779706 6728 ts_tablet_manager.cc:1403] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20251209 03:04:14.779963 6728 raft_consensus.cc:359] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:14.780045 6728 raft_consensus.cc:740] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 37b7f916c7c843dfa479e106f762cfff, State: Initialized, Role: LEARNER
I20251209 03:04:14.780071 6728 consensus_queue.cc:260] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:14.780215 6728 ts_tablet_manager.cc:1434] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:14.780614 6265 tablet_copy_service.cc:342] P b2fc3e20641e45a4b62732385fd4e036: Request end of tablet copy session 37b7f916c7c843dfa479e106f762cfff-439ea3fec68444aeb8cc383b977a7f48 received from {username='slave'} at 127.0.0.1:35124
I20251209 03:04:14.780668 6265 tablet_copy_service.cc:434] P b2fc3e20641e45a4b62732385fd4e036: ending tablet copy session 37b7f916c7c843dfa479e106f762cfff-439ea3fec68444aeb8cc383b977a7f48 on tablet 439ea3fec68444aeb8cc383b977a7f48 with peer 37b7f916c7c843dfa479e106f762cfff
I20251209 03:04:14.807507 6731 tablet_copy_client.cc:538] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:14.808494 6731 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap starting.
I20251209 03:04:14.813165 6731 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=115 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:14.813371 6731 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap complete.
I20251209 03:04:14.813485 6731 ts_tablet_manager.cc:1403] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20251209 03:04:14.813643 6731 raft_consensus.cc:359] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:14.813750 6731 raft_consensus.cc:740] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Initialized, Role: LEARNER
I20251209 03:04:14.813792 6731 consensus_queue.cc:260] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:14.813916 6731 ts_tablet_manager.cc:1434] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent starting tablet: real 0.000s user 0.003s sys 0.000s
I20251209 03:04:14.814157 6190 tablet_copy_service.cc:342] P a6fcaebe8f1f4a319f00f9666d8ba592: Request end of tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-cf139ecb17be4ec58d5c23e86eab88a4 received from {username='slave'} at 127.0.0.1:43178
I20251209 03:04:14.814204 6190 tablet_copy_service.cc:434] P a6fcaebe8f1f4a319f00f9666d8ba592: ending tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-cf139ecb17be4ec58d5c23e86eab88a4 on tablet cf139ecb17be4ec58d5c23e86eab88a4 with peer 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:14.826256 6496 raft_consensus.cc:1217] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251209 03:04:14.878819 6735 tablet_copy_client.cc:538] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:14.879619 6735 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap starting.
I20251209 03:04:14.882755 6738 tablet_copy_client.cc:538] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:14.883597 6738 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap starting.
I20251209 03:04:14.883708 6735 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=99 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:14.883890 6735 tablet_bootstrap.cc:492] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap complete.
I20251209 03:04:14.884074 6735 ts_tablet_manager.cc:1403] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: Time spent bootstrapping tablet: real 0.004s user 0.005s sys 0.000s
I20251209 03:04:14.884281 6735 raft_consensus.cc:359] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:14.884405 6735 raft_consensus.cc:740] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2551ae093c914660b7f6e42bbea920fa, State: Initialized, Role: LEARNER
I20251209 03:04:14.884455 6735 consensus_queue.cc:260] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:14.884569 6735 ts_tablet_manager.cc:1434] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:14.884783 6114 tablet_copy_service.cc:342] P d36845214167428eb93883ecbe8f5385: Request end of tablet copy session 2551ae093c914660b7f6e42bbea920fa-7ead69a2942346dca25124b075136d12 received from {username='slave'} at 127.0.0.1:58920
I20251209 03:04:14.884845 6114 tablet_copy_service.cc:434] P d36845214167428eb93883ecbe8f5385: ending tablet copy session 2551ae093c914660b7f6e42bbea920fa-7ead69a2942346dca25124b075136d12 on tablet 7ead69a2942346dca25124b075136d12 with peer 2551ae093c914660b7f6e42bbea920fa
I20251209 03:04:14.887564 6738 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=130 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:14.887753 6738 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap complete.
I20251209 03:04:14.887859 6738 ts_tablet_manager.cc:1403] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251209 03:04:14.888003 6738 raft_consensus.cc:359] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:14.888078 6738 raft_consensus.cc:740] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Initialized, Role: LEARNER
I20251209 03:04:14.888108 6738 consensus_queue.cc:260] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:14.888181 6738 ts_tablet_manager.cc:1434] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:14.888366 6190 tablet_copy_service.cc:342] P a6fcaebe8f1f4a319f00f9666d8ba592: Request end of tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-f71e8ee2ce1242608207622b267d96ae received from {username='slave'} at 127.0.0.1:43178
I20251209 03:04:14.888417 6190 tablet_copy_service.cc:434] P a6fcaebe8f1f4a319f00f9666d8ba592: ending tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-f71e8ee2ce1242608207622b267d96ae on tablet f71e8ee2ce1242608207622b267d96ae with peer 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:14.890467 6740 tablet_copy_client.cc:538] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:14.891141 6740 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap starting.
I20251209 03:04:14.894456 6740 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=113 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:14.894778 6740 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap complete.
I20251209 03:04:14.894893 6740 ts_tablet_manager.cc:1403] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251209 03:04:14.895044 6740 raft_consensus.cc:359] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:14.895105 6740 raft_consensus.cc:740] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 37b7f916c7c843dfa479e106f762cfff, State: Initialized, Role: LEARNER
I20251209 03:04:14.895133 6740 consensus_queue.cc:260] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:14.895251 6740 ts_tablet_manager.cc:1434] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251209 03:04:14.895458 6190 tablet_copy_service.cc:342] P a6fcaebe8f1f4a319f00f9666d8ba592: Request end of tablet copy session 37b7f916c7c843dfa479e106f762cfff-a9a5e7d1a2de4913a09dfeb084d79ae1 received from {username='slave'} at 127.0.0.1:43190
I20251209 03:04:14.895505 6190 tablet_copy_service.cc:434] P a6fcaebe8f1f4a319f00f9666d8ba592: ending tablet copy session 37b7f916c7c843dfa479e106f762cfff-a9a5e7d1a2de4913a09dfeb084d79ae1 on tablet a9a5e7d1a2de4913a09dfeb084d79ae1 with peer 37b7f916c7c843dfa479e106f762cfff
I20251209 03:04:14.904539 6647 raft_consensus.cc:1217] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.29->[1.30-1.30] Dedup: 1.30->[]
I20251209 03:04:14.965569 6572 raft_consensus.cc:1217] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251209 03:04:15.028203 6496 raft_consensus.cc:1217] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.28->[1.29-1.29] Dedup: 1.29->[]
I20251209 03:04:15.029609 6572 raft_consensus.cc:1217] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.29->[1.30-1.30] Dedup: 1.30->[]
I20251209 03:04:15.204855 6758 raft_consensus.cc:1064] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: attempting to promote NON_VOTER 37b7f916c7c843dfa479e106f762cfff to VOTER
I20251209 03:04:15.205142 6758 consensus_queue.cc:237] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:15.205634 6180 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.205690 6100 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.205802 6496 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.205874 6757 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.205982 6691 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.206159 6760 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.206828 6691 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.207791 6100 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.207906 6496 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.208745 5998 catalog_manager.cc:5654] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:15.209197 6180 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.210579 6252 consensus_queue.cc:237] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:15.210932 6496 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.210951 6100 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.211104 6758 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.211270 6760 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.211687 6758 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.211894 6496 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.211903 6100 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.212711 6001 catalog_manager.cc:5654] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 30 to 31, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.212790 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 439ea3fec68444aeb8cc383b977a7f48 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:15.213511 6170 tablet_service.cc:1552] Processing DeleteTablet for tablet 439ea3fec68444aeb8cc383b977a7f48 with delete_type TABLET_DATA_TOMBSTONED (TS a6fcaebe8f1f4a319f00f9666d8ba592 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:15.213721 6768 tablet_replica.cc:333] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:15.213815 6768 raft_consensus.cc:2243] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:15.213897 6768 raft_consensus.cc:2272] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:15.214118 6768 ts_tablet_manager.cc:1916] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:15.215476 6768 ts_tablet_manager.cc:1929] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251209 03:04:15.215538 6768 log.cc:1199] T 439ea3fec68444aeb8cc383b977a7f48 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/439ea3fec68444aeb8cc383b977a7f48
I20251209 03:04:15.215847 5987 catalog_manager.cc:4985] TS a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307): tablet 439ea3fec68444aeb8cc383b977a7f48 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:15.239779 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:15.248652 6100 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:58930
I20251209 03:04:15.248780 6100 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 2 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:15.251932 6180 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:15.252023 6180 raft_consensus.cc:606] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Received request to transfer leadership to TS b2fc3e20641e45a4b62732385fd4e036
I20251209 03:04:15.252169 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251209 03:04:15.252264 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:15.369737 6647 raft_consensus.cc:1217] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.29->[1.30-1.30] Dedup: 1.30->[]
I20251209 03:04:15.385057 6744 raft_consensus.cc:1064] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: attempting to promote NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e to VOTER
I20251209 03:04:15.385258 6744 consensus_queue.cc:237] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:15.385726 6647 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.385802 6100 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.385888 6250 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.385967 6775 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.386154 6744 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.386278 6774 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.386909 6774 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.387010 6647 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.387981 5998 catalog_manager.cc:5654] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.388228 6250 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.387985 6100 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.389894 6180 consensus_queue.cc:237] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:15.390232 6647 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:15.390408 6100 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:15.390408 6772 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:15.390666 6772 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:15.390980 6772 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.391070 6647 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.391270 6100 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.392166 5998 catalog_manager.cc:5654] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 31 to 32, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.392867 6240 tablet_service.cc:1552] Processing DeleteTablet for tablet f71e8ee2ce1242608207622b267d96ae with delete_type TABLET_DATA_TOMBSTONED (TS b2fc3e20641e45a4b62732385fd4e036 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:35054
I20251209 03:04:15.393023 6779 tablet_replica.cc:333] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:15.393075 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet f71e8ee2ce1242608207622b267d96ae with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:15.393133 6779 raft_consensus.cc:2243] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:15.393225 6779 raft_consensus.cc:2272] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:15.393486 6779 ts_tablet_manager.cc:1916] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:15.394802 6779 ts_tablet_manager.cc:1929] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.31
I20251209 03:04:15.394851 6779 log.cc:1199] T f71e8ee2ce1242608207622b267d96ae P b2fc3e20641e45a4b62732385fd4e036: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/f71e8ee2ce1242608207622b267d96ae
I20251209 03:04:15.395121 5987 catalog_manager.cc:4985] TS b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597): tablet f71e8ee2ce1242608207622b267d96ae (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:15.403049 6747 raft_consensus.cc:1064] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: attempting to promote NON_VOTER 2551ae093c914660b7f6e42bbea920fa to VOTER
I20251209 03:04:15.403254 6747 consensus_queue.cc:237] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:15.403668 6572 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.403712 6250 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.403723 6180 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.403891 6780 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.403995 6747 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.404076 6783 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.404999 6780 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.405078 6250 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.405092 6572 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.406243 5999 catalog_manager.cc:5654] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa reported cstate change: config changed from index 29 to 30, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.406734 6180 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.407805 6100 consensus_queue.cc:237] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:15.407991 6758 raft_consensus.cc:1064] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: attempting to promote NON_VOTER 2551ae093c914660b7f6e42bbea920fa to VOTER
I20251209 03:04:15.408118 6758 consensus_queue.cc:237] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:15.408162 6180 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.408269 6572 raft_consensus.cc:1275] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.408345 6783 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.408445 6747 consensus_queue.cc:1048] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.408530 6572 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.408674 6757 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.408684 6180 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.408689 6100 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.408831 6757 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.408946 6758 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.409420 6747 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.409713 6571 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.409799 6175 raft_consensus.cc:2955] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.410413 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 7ead69a2942346dca25124b075136d12 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:15.410418 6758 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.410778 5998 catalog_manager.cc:5654] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 30 to 31, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.410802 6100 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.411316 6175 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.411777 6571 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.412089 6001 catalog_manager.cc:5654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 30 to 31, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.412358 6240 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ead69a2942346dca25124b075136d12 with delete_type TABLET_DATA_TOMBSTONED (TS b2fc3e20641e45a4b62732385fd4e036 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:35054
I20251209 03:04:15.412556 6779 tablet_replica.cc:333] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:15.412621 6779 raft_consensus.cc:2243] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:15.412688 6779 raft_consensus.cc:2272] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:15.412955 6779 ts_tablet_manager.cc:1916] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:15.413272 6250 consensus_queue.cc:237] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:15.413637 6571 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:15.413812 6691 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:15.413890 6100 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:15.414211 6691 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:15.414268 6779 ts_tablet_manager.cc:1929] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251209 03:04:15.414397 6779 log.cc:1199] T 7ead69a2942346dca25124b075136d12 P b2fc3e20641e45a4b62732385fd4e036: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/7ead69a2942346dca25124b075136d12
I20251209 03:04:15.414723 5987 catalog_manager.cc:4985] TS b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597): tablet 7ead69a2942346dca25124b075136d12 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:15.414776 6760 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.414889 6100 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.415598 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:15.415771 5999 catalog_manager.cc:5654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index 31 to 32, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New cstate: current_term: 1 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:15.416194 6571 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:15.416440 6170 tablet_service.cc:1552] Processing DeleteTablet for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe with delete_type TABLET_DATA_TOMBSTONED (TS a6fcaebe8f1f4a319f00f9666d8ba592 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:15.416550 6768 tablet_replica.cc:333] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:15.416607 6768 raft_consensus.cc:2243] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:15.416666 6768 raft_consensus.cc:2272] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:15.416869 6768 ts_tablet_manager.cc:1916] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:15.418081 6768 ts_tablet_manager.cc:1929] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.31
I20251209 03:04:15.418133 6768 log.cc:1199] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/2beec1d9e1ff42cea8ff5a8e9024a6fe
I20251209 03:04:15.418413 5987 catalog_manager.cc:4985] TS a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307): tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:15.460661 6772 raft_consensus.cc:1064] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: attempting to promote NON_VOTER 37b7f916c7c843dfa479e106f762cfff to VOTER
I20251209 03:04:15.460875 6772 consensus_queue.cc:237] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:15.461375 6496 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.461388 6250 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.461417 6100 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:15.461597 6744 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.461714 6775 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.461820 6774 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:15.462548 6774 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.462656 6250 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.462823 6100 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.463896 5997 catalog_manager.cc:5654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:15.463932 6496 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index 29 to 30, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.464682 6175 consensus_queue.cc:237] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:15.465293 6496 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.465461 6775 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.465932 6250 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.466028 6775 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.466081 6744 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.466372 6250 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.466969 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 with cas_config_opid_index 30: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:15.467022 6496 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:15.467763 5999 catalog_manager.cc:5654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 30 to 31, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:15.468393 6086 tablet_service.cc:1552] Processing DeleteTablet for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 with delete_type TABLET_DATA_TOMBSTONED (TS d36845214167428eb93883ecbe8f5385 not found in new config with opid_index 31) from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:15.468667 6795 tablet_replica.cc:333] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:15.468737 6795 raft_consensus.cc:2243] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:15.468819 6795 raft_consensus.cc:2272] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:15.469082 6795 ts_tablet_manager.cc:1916] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:15.470276 6795 ts_tablet_manager.cc:1929] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.30
I20251209 03:04:15.470347 6795 log.cc:1199] T a9a5e7d1a2de4913a09dfeb084d79ae1 P d36845214167428eb93883ecbe8f5385: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/a9a5e7d1a2de4913a09dfeb084d79ae1
I20251209 03:04:15.470638 5987 catalog_manager.cc:4985] TS d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899): tablet a9a5e7d1a2de4913a09dfeb084d79ae1 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:15.586031 6744 raft_consensus.cc:993] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:15.586135 6744 raft_consensus.cc:1081] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Signalling peer b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:15.586338 6250 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:35060
I20251209 03:04:15.586427 6250 raft_consensus.cc:493] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:15.586457 6250 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:15.587304 6250 raft_consensus.cc:515] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:15.587445 6250 leader_election.cc:290] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 2 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:15.587574 6100 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:15.587615 6175 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:15.587669 6100 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:15.587680 6175 raft_consensus.cc:3055] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:15.587705 6175 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:15.587759 6175 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:15.587829 6175 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:15.588267 6100 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 2.
I20251209 03:04:15.588394 6175 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 2.
I20251209 03:04:15.588505 6220 leader_election.cc:304] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:15.588601 6760 raft_consensus.cc:2804] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:15.588680 6760 raft_consensus.cc:697] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:15.588732 6760 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:15.589239 5997 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 1 to 2, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 2 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } }
I20251209 03:04:15.790212 6780 raft_consensus.cc:993] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:15.790320 6780 raft_consensus.cc:1081] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 2 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:15.790607 6175 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43156
I20251209 03:04:15.790699 6175 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:15.790737 6175 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:15.791603 6175 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:15.791733 6175 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 3 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:15.792006 6100 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:15.792081 6100 raft_consensus.cc:3055] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 2 LEADER]: Stepping down as leader of term 2
I20251209 03:04:15.792104 6100 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:15.792150 6100 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:15.792230 6100 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:15.792744 6100 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 3.
I20251209 03:04:15.792836 6250 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:15.792888 6250 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:15.792959 6145 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:15.793074 6774 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:04:15.793154 6774 raft_consensus.cc:697] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:15.793208 6774 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } }
I20251209 03:04:15.793468 6250 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 3.
I20251209 03:04:15.793754 5997 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 2 to 3, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 3 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } }
I20251209 03:04:15.867209 6744 raft_consensus.cc:1064] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: attempting to promote NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e to VOTER
I20251209 03:04:15.867424 6744 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:15.867900 6250 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.867890 6100 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.867897 6647 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:15.868105 6775 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.868271 6775 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.868373 6772 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:15.869014 6775 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.869172 6647 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.870185 5997 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.870246 6250 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.870289 6100 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
W20251209 03:04:15.871272 6744 consensus_queue.cc:1175] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d36845214167428eb93883ecbe8f5385" responder_term: 1 status { last_received { term: 1 index: 31 } last_committed_idx: 31 last_received_current_leader { term: 1 index: 31 } }
I20251209 03:04:15.871354 6175 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:15.871719 6647 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:15.871729 6250 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:15.871953 6744 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:15.872042 6775 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:15.872517 6775 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.872622 6250 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.872818 6647 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:15.873283 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet cf139ecb17be4ec58d5c23e86eab88a4 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:15.873556 5998 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 31 to 32, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:15.874194 6086 tablet_service.cc:1552] Processing DeleteTablet for tablet cf139ecb17be4ec58d5c23e86eab88a4 with delete_type TABLET_DATA_TOMBSTONED (TS d36845214167428eb93883ecbe8f5385 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:15.874291 6795 tablet_replica.cc:333] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:15.874364 6795 raft_consensus.cc:2243] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:15.874436 6795 raft_consensus.cc:2272] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:15.874670 6795 ts_tablet_manager.cc:1916] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:15.875696 6795 ts_tablet_manager.cc:1929] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.31
I20251209 03:04:15.875743 6795 log.cc:1199] T cf139ecb17be4ec58d5c23e86eab88a4 P d36845214167428eb93883ecbe8f5385: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/cf139ecb17be4ec58d5c23e86eab88a4
I20251209 03:04:15.876016 5987 catalog_manager.cc:4985] TS d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899): tablet cf139ecb17be4ec58d5c23e86eab88a4 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:16.150244 6100 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251209 03:04:16.150532 6760 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:16.151886 6180 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251209 03:04:16.152073 6805 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:16.199417 6250 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251209 03:04:16.199738 6772 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:16.201030 6100 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251209 03:04:16.201361 6772 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:16.874590 6180 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:16.875106 6100 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251209 03:04:16.875152 6250 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
W20251209 03:04:16.875208 6143 consensus_peers.cc:597] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979): Couldn't send request to peer 37b7f916c7c843dfa479e106f762cfff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ef8ec5789d64b8f931598cfcded95f2. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:16.875325 6772 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:16.875427 6812 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:16.876004 6811 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Committing config change with OpId 3.32: config changed from index -1 to 32, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:16.876158 6250 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index -1 to 32, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:16.876156 6100 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index -1 to 32, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:16.877380 5999 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index -1 to 32, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New cstate: current_term: 3 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:16.877435 6099 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:16.878408 6250 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:16.878446 6180 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:16.878589 6815 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:16.878733 6817 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:16.880045 6815 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:16.881306 6175 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:16.881364 6180 consensus_queue.cc:237] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
W20251209 03:04:16.881557 6070 consensus_peers.cc:597] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 -> Peer 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151): Couldn't send request to peer 03aecf9a33ec4b4ba130772d3c35d33e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b74f825019240c7950dcc767dd2e6a5. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:16.881916 5997 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:16.881995 6496 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:16.882002 6250 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:16.882170 6812 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:16.882298 6811 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:16.882467 6250 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:16.882956 6812 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:16.883668 6252 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:16.884140 6100 consensus_queue.cc:237] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:16.884594 6175 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:16.884663 6496 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:16.884825 6747 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
W20251209 03:04:16.884928 6070 consensus_peers.cc:597] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 -> Peer 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151): Couldn't send request to peer 03aecf9a33ec4b4ba130772d3c35d33e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86b92235e7ad4a959951bdb816e39905. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:16.885107 5998 catalog_manager.cc:5654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 31 to 32, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:16.885196 6252 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 1 index: 30. (index mismatch)
I20251209 03:04:16.885370 6747 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
W20251209 03:04:16.885938 6146 consensus_peers.cc:597] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607): Couldn't send request to peer 2551ae093c914660b7f6e42bbea920fa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9a5e7d1a2de4913a09dfeb084d79ae1. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:16.886139 6817 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:16.886315 6252 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:16.886300 6175 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.30: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:16.887255 6175 consensus_queue.cc:237] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:16.887251 5998 catalog_manager.cc:5654] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index -1 to 30, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:16.887737 6647 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251209 03:04:16.887763 6100 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
W20251209 03:04:16.888020 6143 consensus_peers.cc:597] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979): Couldn't send request to peer 37b7f916c7c843dfa479e106f762cfff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f71e8ee2ce1242608207622b267d96ae. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:16.888020 6811 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:16.888154 6812 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:16.888768 6812 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:16.888885 6100 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:16.888952 6647 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:16.889882 5999 catalog_manager.cc:5654] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 32 to 33, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:16.889935 6175 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:16.890350 6647 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
I20251209 03:04:16.890447 6252 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 1 index: 33. (index mismatch)
W20251209 03:04:16.890487 6146 consensus_peers.cc:597] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607): Couldn't send request to peer 2551ae093c914660b7f6e42bbea920fa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf139ecb17be4ec58d5c23e86eab88a4. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:16.890633 6812 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:16.890794 6811 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:16.891176 6812 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:16.891275 6647 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:16.891510 6252 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.33: config changed from index 32 to 33, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:16.892288 5997 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 32 to 33, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:17.252393 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:17.252843 6100 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "0e6b3f62d7894c69b4cf99bf1ad942ce"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:58930
I20251209 03:04:17.252954 6100 raft_consensus.cc:606] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:17.253515 6100 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
new_leader_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:58930
I20251209 03:04:17.253595 6100 raft_consensus.cc:606] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Received request to transfer leadership to TS b2fc3e20641e45a4b62732385fd4e036
I20251209 03:04:17.253888 6252 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2beec1d9e1ff42cea8ff5a8e9024a6fe"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "2551ae093c914660b7f6e42bbea920fa"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:17.253954 6252 raft_consensus.cc:606] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Received request to transfer leadership to TS 2551ae093c914660b7f6e42bbea920fa
I20251209 03:04:17.254189 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "a9a5e7d1a2de4913a09dfeb084d79ae1"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "37b7f916c7c843dfa479e106f762cfff"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:17.254261 6175 raft_consensus.cc:606] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Received request to transfer leadership to TS 37b7f916c7c843dfa479e106f762cfff
I20251209 03:04:17.254518 6252 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:17.254587 6252 raft_consensus.cc:606] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:17.254863 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "cf139ecb17be4ec58d5c23e86eab88a4"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:17.254915 6175 raft_consensus.cc:606] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Received request to transfer leadership to TS 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:17.255115 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20251209 03:04:17.255198 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:17.263311 6805 raft_consensus.cc:993] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower 2551ae093c914660b7f6e42bbea920fa to start an election
I20251209 03:04:17.263417 6805 raft_consensus.cc:1081] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Signalling peer 2551ae093c914660b7f6e42bbea920fa to start an election
I20251209 03:04:17.263808 6571 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "2beec1d9e1ff42cea8ff5a8e9024a6fe"
dest_uuid: "2551ae093c914660b7f6e42bbea920fa"
from {username='slave'} at 127.0.0.1:45352
I20251209 03:04:17.263911 6571 raft_consensus.cc:493] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:17.263944 6571 raft_consensus.cc:3060] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.265785 6842 ts_tablet_manager.cc:933] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: Initiating tablet copy from peer d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899)
I20251209 03:04:17.265940 6842 tablet_copy_client.cc:323] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.193:43899
I20251209 03:04:17.266001 6571 raft_consensus.cc:515] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:17.266145 6571 leader_election.cc:290] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [CANDIDATE]: Term 2 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:17.266306 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2beec1d9e1ff42cea8ff5a8e9024a6fe" candidate_uuid: "2551ae093c914660b7f6e42bbea920fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:17.266381 6099 raft_consensus.cc:3060] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.267351 6099 raft_consensus.cc:2468] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2551ae093c914660b7f6e42bbea920fa in term 2.
I20251209 03:04:17.267439 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2beec1d9e1ff42cea8ff5a8e9024a6fe" candidate_uuid: "2551ae093c914660b7f6e42bbea920fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:17.267493 6252 raft_consensus.cc:3055] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:17.267519 6252 raft_consensus.cc:740] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:17.267582 6252 consensus_queue.cc:260] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:17.267663 6252 raft_consensus.cc:3060] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.268280 6252 raft_consensus.cc:2468] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2551ae093c914660b7f6e42bbea920fa in term 2.
I20251209 03:04:17.268443 6537 leader_election.cc:304] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2551ae093c914660b7f6e42bbea920fa, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:17.268625 6846 raft_consensus.cc:2804] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:17.268743 6846 raft_consensus.cc:697] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 2 LEADER]: Becoming Leader. State: Replica: 2551ae093c914660b7f6e42bbea920fa, State: Running, Role: LEADER
I20251209 03:04:17.268821 6846 consensus_queue.cc:237] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:17.269295 5999 catalog_manager.cc:5654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa reported cstate change: term changed from 1 to 2, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to 2551ae093c914660b7f6e42bbea920fa (127.2.170.197). New cstate: current_term: 2 leader_uuid: "2551ae093c914660b7f6e42bbea920fa" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:17.271670 6114 tablet_copy_service.cc:140] P d36845214167428eb93883ecbe8f5385: Received BeginTabletCopySession request for tablet 86b92235e7ad4a959951bdb816e39905 from peer 03aecf9a33ec4b4ba130772d3c35d33e ({username='slave'} at 127.0.0.1:58938)
I20251209 03:04:17.271728 6114 tablet_copy_service.cc:161] P d36845214167428eb93883ecbe8f5385: Beginning new tablet copy session on tablet 86b92235e7ad4a959951bdb816e39905 from peer 03aecf9a33ec4b4ba130772d3c35d33e at {username='slave'} at 127.0.0.1:58938: session id = 03aecf9a33ec4b4ba130772d3c35d33e-86b92235e7ad4a959951bdb816e39905
I20251209 03:04:17.272269 6114 tablet_copy_source_session.cc:215] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:17.272580 6842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b92235e7ad4a959951bdb816e39905. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:17.273998 6842 tablet_copy_client.cc:806] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:17.274108 6842 tablet_copy_client.cc:670] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:17.274591 6843 tablet_copy_client.cc:1025] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:17.275926 6849 ts_tablet_manager.cc:933] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: Initiating tablet copy from peer a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:17.276072 6850 ts_tablet_manager.cc:933] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: Initiating tablet copy from peer d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899)
I20251209 03:04:17.276082 6849 tablet_copy_client.cc:323] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42307
I20251209 03:04:17.276243 6850 tablet_copy_client.cc:323] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.193:43899
I20251209 03:04:17.276257 6190 tablet_copy_service.cc:140] P a6fcaebe8f1f4a319f00f9666d8ba592: Received BeginTabletCopySession request for tablet f71e8ee2ce1242608207622b267d96ae from peer 37b7f916c7c843dfa479e106f762cfff ({username='slave'} at 127.0.0.1:43190)
I20251209 03:04:17.276346 6190 tablet_copy_service.cc:161] P a6fcaebe8f1f4a319f00f9666d8ba592: Beginning new tablet copy session on tablet f71e8ee2ce1242608207622b267d96ae from peer 37b7f916c7c843dfa479e106f762cfff at {username='slave'} at 127.0.0.1:43190: session id = 37b7f916c7c843dfa479e106f762cfff-f71e8ee2ce1242608207622b267d96ae
I20251209 03:04:17.276397 6114 tablet_copy_service.cc:140] P d36845214167428eb93883ecbe8f5385: Received BeginTabletCopySession request for tablet 2b74f825019240c7950dcc767dd2e6a5 from peer 03aecf9a33ec4b4ba130772d3c35d33e ({username='slave'} at 127.0.0.1:58938)
I20251209 03:04:17.276433 6114 tablet_copy_service.cc:161] P d36845214167428eb93883ecbe8f5385: Beginning new tablet copy session on tablet 2b74f825019240c7950dcc767dd2e6a5 from peer 03aecf9a33ec4b4ba130772d3c35d33e at {username='slave'} at 127.0.0.1:58938: session id = 03aecf9a33ec4b4ba130772d3c35d33e-2b74f825019240c7950dcc767dd2e6a5
I20251209 03:04:17.276862 6190 tablet_copy_source_session.cc:215] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:17.277163 6849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71e8ee2ce1242608207622b267d96ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:17.277280 6114 tablet_copy_source_session.cc:215] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:17.277760 6850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b74f825019240c7950dcc767dd2e6a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:17.278544 6849 tablet_copy_client.cc:806] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:17.278645 6849 tablet_copy_client.cc:670] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:17.279055 6850 tablet_copy_client.cc:806] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:17.279119 6851 tablet_copy_client.cc:1025] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:17.279234 6850 tablet_copy_client.cc:670] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:17.279781 6852 tablet_copy_client.cc:1025] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:17.360749 6853 ts_tablet_manager.cc:933] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: Initiating tablet copy from peer a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:17.361048 6853 tablet_copy_client.cc:323] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42307
I20251209 03:04:17.361267 6190 tablet_copy_service.cc:140] P a6fcaebe8f1f4a319f00f9666d8ba592: Received BeginTabletCopySession request for tablet 5ef8ec5789d64b8f931598cfcded95f2 from peer 37b7f916c7c843dfa479e106f762cfff ({username='slave'} at 127.0.0.1:43190)
I20251209 03:04:17.361346 6190 tablet_copy_service.cc:161] P a6fcaebe8f1f4a319f00f9666d8ba592: Beginning new tablet copy session on tablet 5ef8ec5789d64b8f931598cfcded95f2 from peer 37b7f916c7c843dfa479e106f762cfff at {username='slave'} at 127.0.0.1:43190: session id = 37b7f916c7c843dfa479e106f762cfff-5ef8ec5789d64b8f931598cfcded95f2
I20251209 03:04:17.361829 6190 tablet_copy_source_session.cc:215] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:17.362221 6853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ef8ec5789d64b8f931598cfcded95f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:17.363719 6853 tablet_copy_client.cc:806] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:17.363828 6853 tablet_copy_client.cc:670] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:17.364342 6854 tablet_copy_client.cc:1025] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:17.406792 6811 raft_consensus.cc:993] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower 37b7f916c7c843dfa479e106f762cfff to start an election
I20251209 03:04:17.406886 6811 raft_consensus.cc:1081] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Signalling peer 37b7f916c7c843dfa479e106f762cfff to start an election
I20251209 03:04:17.407066 6496 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "a9a5e7d1a2de4913a09dfeb084d79ae1"
dest_uuid: "37b7f916c7c843dfa479e106f762cfff"
from {username='slave'} at 127.0.0.1:41522
I20251209 03:04:17.407142 6496 raft_consensus.cc:493] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:17.407171 6496 raft_consensus.cc:3060] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.407940 6496 raft_consensus.cc:515] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:17.408106 6496 leader_election.cc:290] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [CANDIDATE]: Term 2 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:17.408293 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9a5e7d1a2de4913a09dfeb084d79ae1" candidate_uuid: "37b7f916c7c843dfa479e106f762cfff" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:17.408264 6175 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9a5e7d1a2de4913a09dfeb084d79ae1" candidate_uuid: "37b7f916c7c843dfa479e106f762cfff" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:17.408394 6175 raft_consensus.cc:3055] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:17.408385 6252 raft_consensus.cc:3060] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.408456 6175 raft_consensus.cc:740] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:17.408504 6175 consensus_queue.cc:260] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:17.408598 6175 raft_consensus.cc:3060] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.408986 6252 raft_consensus.cc:2468] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37b7f916c7c843dfa479e106f762cfff in term 2.
I20251209 03:04:17.409080 6175 raft_consensus.cc:2468] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37b7f916c7c843dfa479e106f762cfff in term 2.
I20251209 03:04:17.409147 6461 leader_election.cc:304] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b7f916c7c843dfa479e106f762cfff, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:17.409291 6856 raft_consensus.cc:2804] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:17.409379 6856 raft_consensus.cc:697] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Becoming Leader. State: Replica: 37b7f916c7c843dfa479e106f762cfff, State: Running, Role: LEADER
I20251209 03:04:17.409428 6856 consensus_queue.cc:237] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:17.410102 5999 catalog_manager.cc:5654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff reported cstate change: term changed from 1 to 2, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to 37b7f916c7c843dfa479e106f762cfff (127.2.170.196). New cstate: current_term: 2 leader_uuid: "37b7f916c7c843dfa479e106f762cfff" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:17.456856 6860 ts_tablet_manager.cc:933] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: Initiating tablet copy from peer a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:17.457086 6860 tablet_copy_client.cc:323] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42307
I20251209 03:04:17.460284 6190 tablet_copy_service.cc:140] P a6fcaebe8f1f4a319f00f9666d8ba592: Received BeginTabletCopySession request for tablet cf139ecb17be4ec58d5c23e86eab88a4 from peer 2551ae093c914660b7f6e42bbea920fa ({username='slave'} at 127.0.0.1:43212)
I20251209 03:04:17.460361 6190 tablet_copy_service.cc:161] P a6fcaebe8f1f4a319f00f9666d8ba592: Beginning new tablet copy session on tablet cf139ecb17be4ec58d5c23e86eab88a4 from peer 2551ae093c914660b7f6e42bbea920fa at {username='slave'} at 127.0.0.1:43212: session id = 2551ae093c914660b7f6e42bbea920fa-cf139ecb17be4ec58d5c23e86eab88a4
I20251209 03:04:17.460901 6190 tablet_copy_source_session.cc:215] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:17.461213 6860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf139ecb17be4ec58d5c23e86eab88a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:17.462502 6860 tablet_copy_client.cc:806] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:17.462623 6860 tablet_copy_client.cc:670] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:17.463148 6861 tablet_copy_client.cc:1025] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:17.476455 6855 raft_consensus.cc:993] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower 03aecf9a33ec4b4ba130772d3c35d33e to start an election
I20251209 03:04:17.476542 6811 raft_consensus.cc:1081] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Signalling peer 03aecf9a33ec4b4ba130772d3c35d33e to start an election
I20251209 03:04:17.476701 6647 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "cf139ecb17be4ec58d5c23e86eab88a4"
dest_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
from {username='slave'} at 127.0.0.1:40306
I20251209 03:04:17.476773 6647 raft_consensus.cc:493] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:17.476799 6647 raft_consensus.cc:3060] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.477499 6647 raft_consensus.cc:515] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:17.477663 6647 leader_election.cc:290] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [CANDIDATE]: Term 2 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:17.478065 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf139ecb17be4ec58d5c23e86eab88a4" candidate_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" candidate_term: 2 candidate_status { last_received { term: 1 index: 33 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:17.478149 6180 raft_consensus.cc:3055] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:17.478170 6180 raft_consensus.cc:740] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:17.478224 6180 consensus_queue.cc:260] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:17.478319 6180 raft_consensus.cc:3060] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.479271 6180 raft_consensus.cc:2468] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03aecf9a33ec4b4ba130772d3c35d33e in term 2.
I20251209 03:04:17.479566 6612 leader_election.cc:304] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03aecf9a33ec4b4ba130772d3c35d33e, a6fcaebe8f1f4a319f00f9666d8ba592; no voters:
I20251209 03:04:17.479786 6865 raft_consensus.cc:2804] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:17.479926 6865 raft_consensus.cc:697] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Becoming Leader. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Running, Role: LEADER
I20251209 03:04:17.479993 6865 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:17.480605 5999 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: term changed from 1 to 2, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198). New cstate: current_term: 2 leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:17.480836 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cf139ecb17be4ec58d5c23e86eab88a4" candidate_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" candidate_term: 2 candidate_status { last_received { term: 1 index: 33 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:17.480897 6252 raft_consensus.cc:3060] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.481508 6252 raft_consensus.cc:2468] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03aecf9a33ec4b4ba130772d3c35d33e in term 2.
I20251209 03:04:17.482043 6815 raft_consensus.cc:993] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385: : Instructing follower b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:17.482103 6815 raft_consensus.cc:1081] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Signalling peer b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:17.482251 6252 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:35074
I20251209 03:04:17.482314 6252 raft_consensus.cc:493] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:17.482339 6252 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.482764 6252 raft_consensus.cc:515] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:17.482862 6252 leader_election.cc:290] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 2 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:17.482988 6100 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:17.483026 6175 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 2 candidate_status { last_received { term: 1 index: 30 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:17.483062 6100 raft_consensus.cc:3055] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:17.483067 6175 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.483115 6100 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:17.483183 6100 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:17.483296 6100 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.483755 6175 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 2.
I20251209 03:04:17.483855 6100 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 2.
I20251209 03:04:17.483893 6220 leader_election.cc:304] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:17.484022 6841 raft_consensus.cc:2804] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:17.484073 6841 raft_consensus.cc:697] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:17.484122 6841 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:17.484587 5999 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 1 to 2, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 2 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:17.536873 6841 raft_consensus.cc:993] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:17.536974 6841 raft_consensus.cc:1081] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:17.537195 6175 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43158
I20251209 03:04:17.537297 6175 raft_consensus.cc:493] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:17.537326 6175 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:17.538149 6175 raft_consensus.cc:515] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:17.538277 6175 leader_election.cc:290] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 3 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:17.538357 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:17.538380 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 3 candidate_status { last_received { term: 2 index: 30 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:17.538426 6099 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:17.538431 6252 raft_consensus.cc:3055] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Stepping down as leader of term 2
I20251209 03:04:17.538467 6252 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:17.538524 6252 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:17.538602 6252 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:17.539099 6252 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 3.
I20251209 03:04:17.539099 6099 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 3.
I20251209 03:04:17.539259 6145 leader_election.cc:304] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:17.539350 6811 raft_consensus.cc:2804] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:04:17.539402 6811 raft_consensus.cc:697] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:17.539441 6811 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 30, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:17.540042 5999 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 2 to 3, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 3 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:17.611980 6815 raft_consensus.cc:993] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:17.612066 6815 raft_consensus.cc:1081] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:17.612397 6175 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "0e6b3f62d7894c69b4cf99bf1ad942ce"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43156
I20251209 03:04:17.612495 6175 raft_consensus.cc:493] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:17.612528 6175 raft_consensus.cc:3060] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.613507 6175 raft_consensus.cc:515] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:17.613642 6175 leader_election.cc:290] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 2 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:17.613778 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e6b3f62d7894c69b4cf99bf1ad942ce" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:17.613834 6099 raft_consensus.cc:3055] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:17.613838 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e6b3f62d7894c69b4cf99bf1ad942ce" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 2 candidate_status { last_received { term: 1 index: 29 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:17.614044 6252 raft_consensus.cc:3060] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.614913 6252 raft_consensus.cc:2468] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 2.
I20251209 03:04:17.613868 6099 raft_consensus.cc:740] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:17.615072 6145 leader_election.cc:304] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:17.615161 6811 raft_consensus.cc:2804] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:17.615075 6099 consensus_queue.cc:260] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:17.615228 6811 raft_consensus.cc:697] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:17.615257 6099 raft_consensus.cc:3060] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:17.615276 6811 consensus_queue.cc:237] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 1.29, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:17.615898 6099 raft_consensus.cc:2468] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 2.
I20251209 03:04:17.616109 5999 catalog_manager.cc:5654] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 1 to 2, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 2 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
W20251209 03:04:17.814210 6462 consensus_peers.cc:597] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff -> Peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607): Couldn't send request to peer 2551ae093c914660b7f6e42bbea920fa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9a5e7d1a2de4913a09dfeb084d79ae1. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:17.823647 6099 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:17.823853 6874 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:17.825006 6252 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:17.825220 6877 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:17.852640 6175 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:17.852829 6856 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:17.854015 6252 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:17.854197 6885 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:17.861055 6175 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251209 03:04:17.861296 6841 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:17.862205 6099 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251209 03:04:17.862368 6805 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
W20251209 03:04:17.865358 6221 consensus_peers.cc:597] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 -> Peer 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151): Couldn't send request to peer 03aecf9a33ec4b4ba130772d3c35d33e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:17.961063 6180 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:17.961356 6865 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:17.962805 6252 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:17.963059 6866 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
W20251209 03:04:17.965734 6613 consensus_peers.cc:597] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e -> Peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607): Couldn't send request to peer 2551ae093c914660b7f6e42bbea920fa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:17.991714 6250 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251209 03:04:17.991940 6889 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:17.992834 6099 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 29. Preceding OpId from leader: term: 2 index: 30. (index mismatch)
I20251209 03:04:17.993016 6811 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:17.997864 6252 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251209 03:04:17.998027 6889 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:17.999161 6099 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251209 03:04:17.999382 6811 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:18.240710 6898 ts_tablet_manager.cc:933] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: Initiating tablet copy from peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:18.240926 6898 tablet_copy_client.cc:323] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.196:43979
I20251209 03:04:18.244284 6506 tablet_copy_service.cc:140] P 37b7f916c7c843dfa479e106f762cfff: Received BeginTabletCopySession request for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 from peer 2551ae093c914660b7f6e42bbea920fa ({username='slave'} at 127.0.0.1:41540)
I20251209 03:04:18.244347 6506 tablet_copy_service.cc:161] P 37b7f916c7c843dfa479e106f762cfff: Beginning new tablet copy session on tablet a9a5e7d1a2de4913a09dfeb084d79ae1 from peer 2551ae093c914660b7f6e42bbea920fa at {username='slave'} at 127.0.0.1:41540: session id = 2551ae093c914660b7f6e42bbea920fa-a9a5e7d1a2de4913a09dfeb084d79ae1
I20251209 03:04:18.244889 6506 tablet_copy_source_session.cc:215] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:18.245172 6898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a5e7d1a2de4913a09dfeb084d79ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:18.246477 6898 tablet_copy_client.cc:806] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:18.246693 6898 tablet_copy_client.cc:670] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:18.247154 6899 tablet_copy_client.cc:1025] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:19.255321 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:19.255743 6250 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:19.255831 6250 raft_consensus.cc:606] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:19.256129 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:19.256209 6175 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Received request to transfer leadership to TS d36845214167428eb93883ecbe8f5385
I20251209 03:04:19.256541 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:19.256604 6175 raft_consensus.cc:606] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Received request to transfer leadership to TS d36845214167428eb93883ecbe8f5385
I20251209 03:04:19.256745 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251209 03:04:19.256810 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:19.274912 6842 tablet_copy_client.cc:538] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:19.275770 6842 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap starting.
I20251209 03:04:19.279428 6842 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=113 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:19.279527 6849 tablet_copy_client.cc:538] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:19.279654 6842 tablet_bootstrap.cc:492] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap complete.
I20251209 03:04:19.279762 6842 ts_tablet_manager.cc:1403] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20251209 03:04:19.279906 6842 raft_consensus.cc:359] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.279978 6842 raft_consensus.cc:740] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Initialized, Role: LEARNER
I20251209 03:04:19.280064 6850 tablet_copy_client.cc:538] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:19.280102 6842 consensus_queue.cc:260] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.280332 6842 ts_tablet_manager.cc:1434] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:19.280365 6849 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: Bootstrap starting.
I20251209 03:04:19.280864 6850 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap starting.
I20251209 03:04:19.283735 6849 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=130 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:19.283808 6850 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap replayed 1/1 log segments. Stats: ops{read=30 overwritten=0 applied=30 ignored=0} inserts{seen=108 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:19.283911 6849 tablet_bootstrap.cc:492] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: Bootstrap complete.
I20251209 03:04:19.283988 6850 tablet_bootstrap.cc:492] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap complete.
I20251209 03:04:19.284003 6849 ts_tablet_manager.cc:1403] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20251209 03:04:19.284142 6849 raft_consensus.cc:359] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.284214 6849 raft_consensus.cc:740] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 37b7f916c7c843dfa479e106f762cfff, State: Initialized, Role: LEARNER
I20251209 03:04:19.284251 6849 consensus_queue.cc:260] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.284173 6850 ts_tablet_manager.cc:1403] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251209 03:04:19.284420 6850 raft_consensus.cc:359] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.284476 6850 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Initialized, Role: LEARNER
I20251209 03:04:19.284500 6850 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 1.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.284613 6850 ts_tablet_manager.cc:1434] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:19.284785 6114 tablet_copy_service.cc:342] P d36845214167428eb93883ecbe8f5385: Request end of tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-2b74f825019240c7950dcc767dd2e6a5 received from {username='slave'} at 127.0.0.1:58938
I20251209 03:04:19.284832 6114 tablet_copy_service.cc:434] P d36845214167428eb93883ecbe8f5385: ending tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-2b74f825019240c7950dcc767dd2e6a5 on tablet 2b74f825019240c7950dcc767dd2e6a5 with peer 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:19.284730 6849 ts_tablet_manager.cc:1434] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251209 03:04:19.285053 6114 tablet_copy_service.cc:342] P d36845214167428eb93883ecbe8f5385: Request end of tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-86b92235e7ad4a959951bdb816e39905 received from {username='slave'} at 127.0.0.1:58938
I20251209 03:04:19.285104 6114 tablet_copy_service.cc:434] P d36845214167428eb93883ecbe8f5385: ending tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-86b92235e7ad4a959951bdb816e39905 on tablet 86b92235e7ad4a959951bdb816e39905 with peer 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:19.285209 6190 tablet_copy_service.cc:342] P a6fcaebe8f1f4a319f00f9666d8ba592: Request end of tablet copy session 37b7f916c7c843dfa479e106f762cfff-f71e8ee2ce1242608207622b267d96ae received from {username='slave'} at 127.0.0.1:43190
I20251209 03:04:19.285243 6190 tablet_copy_service.cc:434] P a6fcaebe8f1f4a319f00f9666d8ba592: ending tablet copy session 37b7f916c7c843dfa479e106f762cfff-f71e8ee2ce1242608207622b267d96ae on tablet f71e8ee2ce1242608207622b267d96ae with peer 37b7f916c7c843dfa479e106f762cfff
I20251209 03:04:19.310853 6646 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Advancing to term 2
I20251209 03:04:19.344875 6496 raft_consensus.cc:1217] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.32->[1.33-1.33] Dedup: 1.33->[]
I20251209 03:04:19.365365 6853 tablet_copy_client.cc:538] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:19.366418 6853 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap starting.
I20251209 03:04:19.370918 6853 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=32 overwritten=0 applied=32 ignored=0} inserts{seen=110 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:19.371119 6853 tablet_bootstrap.cc:492] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap complete.
I20251209 03:04:19.371245 6853 ts_tablet_manager.cc:1403] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20251209 03:04:19.371433 6853 raft_consensus.cc:359] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.371515 6853 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 37b7f916c7c843dfa479e106f762cfff, State: Initialized, Role: LEARNER
I20251209 03:04:19.371537 6853 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.371661 6853 ts_tablet_manager.cc:1434] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:04:19.371928 6190 tablet_copy_service.cc:342] P a6fcaebe8f1f4a319f00f9666d8ba592: Request end of tablet copy session 37b7f916c7c843dfa479e106f762cfff-5ef8ec5789d64b8f931598cfcded95f2 received from {username='slave'} at 127.0.0.1:43190
I20251209 03:04:19.371987 6190 tablet_copy_service.cc:434] P a6fcaebe8f1f4a319f00f9666d8ba592: ending tablet copy session 37b7f916c7c843dfa479e106f762cfff-5ef8ec5789d64b8f931598cfcded95f2 on tablet 5ef8ec5789d64b8f931598cfcded95f2 with peer 37b7f916c7c843dfa479e106f762cfff
I20251209 03:04:19.381899 6646 raft_consensus.cc:1217] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.29->[1.30-1.30] Dedup: 1.30->[]
I20251209 03:04:19.402833 6904 raft_consensus.cc:993] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:19.402927 6904 raft_consensus.cc:1081] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:19.403081 6175 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43158
I20251209 03:04:19.403158 6175 raft_consensus.cc:493] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:19.403185 6175 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:19.403833 6175 raft_consensus.cc:515] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.403975 6175 leader_election.cc:290] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 3 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:19.404075 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 3 candidate_status { last_received { term: 2 index: 31 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:19.404093 6250 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 3 candidate_status { last_received { term: 2 index: 31 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:19.404142 6099 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:19.404148 6250 raft_consensus.cc:3055] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Stepping down as leader of term 2
I20251209 03:04:19.404183 6250 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:19.404229 6250 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.404311 6250 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:19.404851 6250 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 3.
I20251209 03:04:19.404851 6099 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 3.
I20251209 03:04:19.405014 6145 leader_election.cc:304] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:19.405097 6889 raft_consensus.cc:2804] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:04:19.405236 6889 raft_consensus.cc:697] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:19.405303 6889 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.405843 5998 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 2 to 3, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 3 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:19.407703 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:19.407763 6175 raft_consensus.cc:606] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Received request to transfer leadership
I20251209 03:04:19.409334 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:19.409384 6175 raft_consensus.cc:606] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Received request to transfer leadership
W20251209 03:04:19.409492 5987 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 2b74f825019240c7950dcc767dd2e6a5 already in progress [suppressed 3 similar messages]
I20251209 03:04:19.409771 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:19.409816 6175 raft_consensus.cc:606] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Received request to transfer leadership
W20251209 03:04:19.409937 6049 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 2b74f825019240c7950dcc767dd2e6a5 already in progress
W20251209 03:04:19.409982 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 2b74f825019240c7950dcc767dd2e6a5 already in progress
I20251209 03:04:19.420277 6496 raft_consensus.cc:1217] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 3 LEARNER]: Deduplicated request from leader. Original: 3.31->[3.32-3.32] Dedup: 3.32->[]
I20251209 03:04:19.463469 6860 tablet_copy_client.cc:538] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:19.464449 6860 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap starting.
I20251209 03:04:19.468220 6860 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=115 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:19.468398 6860 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap complete.
I20251209 03:04:19.468537 6860 ts_tablet_manager.cc:1403] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: Time spent bootstrapping tablet: real 0.004s user 0.005s sys 0.000s
I20251209 03:04:19.468698 6860 raft_consensus.cc:359] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:19.468778 6860 raft_consensus.cc:740] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2551ae093c914660b7f6e42bbea920fa, State: Initialized, Role: LEARNER
I20251209 03:04:19.468809 6860 consensus_queue.cc:260] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:19.468921 6860 ts_tablet_manager.cc:1434] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:19.469110 6190 tablet_copy_service.cc:342] P a6fcaebe8f1f4a319f00f9666d8ba592: Request end of tablet copy session 2551ae093c914660b7f6e42bbea920fa-cf139ecb17be4ec58d5c23e86eab88a4 received from {username='slave'} at 127.0.0.1:43212
I20251209 03:04:19.469170 6190 tablet_copy_service.cc:434] P a6fcaebe8f1f4a319f00f9666d8ba592: ending tablet copy session 2551ae093c914660b7f6e42bbea920fa-cf139ecb17be4ec58d5c23e86eab88a4 on tablet cf139ecb17be4ec58d5c23e86eab88a4 with peer 2551ae093c914660b7f6e42bbea920fa
I20251209 03:04:19.478281 6914 raft_consensus.cc:993] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:19.478346 6914 raft_consensus.cc:1081] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Signalling peer d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:19.478520 6099 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:58880
I20251209 03:04:19.478576 6099 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:19.478601 6099 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:19.479115 6099 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.479233 6099 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 4 election: Requested vote from peers b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:19.479374 6250 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:19.479444 6250 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:19.479382 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:19.479539 6180 raft_consensus.cc:3055] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Stepping down as leader of term 3
I20251209 03:04:19.479568 6180 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:19.479612 6180 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.479702 6180 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:19.479948 6250 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 4.
I20251209 03:04:19.480139 6069 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:19.480234 6871 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Leader election won for term 4
I20251209 03:04:19.480333 6871 raft_consensus.cc:697] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:19.480381 6871 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.480417 6180 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 4.
I20251209 03:04:19.480885 5998 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 3 to 4, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 4 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:19.482007 6099 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:19.482059 6099 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Received request to transfer leadership
I20251209 03:04:19.483345 6099 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:19.483397 6099 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Received request to transfer leadership
I20251209 03:04:19.483649 6099 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:19.483707 6099 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Received request to transfer leadership
W20251209 03:04:19.483832 6049 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 5ef8ec5789d64b8f931598cfcded95f2 already in progress
W20251209 03:04:19.483863 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 5ef8ec5789d64b8f931598cfcded95f2 already in progress
I20251209 03:04:19.626538 6889 raft_consensus.cc:993] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:19.626631 6889 raft_consensus.cc:1081] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Signalling peer d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:19.626797 6099 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:58880
I20251209 03:04:19.626875 6099 raft_consensus.cc:493] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:19.626904 6099 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:19.627908 6099 raft_consensus.cc:515] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:19.628053 6099 leader_election.cc:290] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 4 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:19.628198 6250 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:19.628214 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 4 candidate_status { last_received { term: 3 index: 31 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:19.628294 6250 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:19.628304 6180 raft_consensus.cc:3055] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Stepping down as leader of term 3
I20251209 03:04:19.628340 6180 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:19.628391 6180 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:19.628482 6180 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:19.628983 6180 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 4.
I20251209 03:04:19.629067 6250 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 4.
I20251209 03:04:19.629114 6069 leader_election.cc:304] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:19.629179 6871 raft_consensus.cc:2804] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Leader election won for term 4
I20251209 03:04:19.629225 6871 raft_consensus.cc:697] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:19.629274 6871 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:19.629824 5998 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 3 to 4, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 4 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:19.791970 6925 raft_consensus.cc:1064] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: attempting to promote NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e to VOTER
I20251209 03:04:19.792160 6925 consensus_queue.cc:237] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 1.30, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:19.792572 6646 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 LEARNER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:19.792590 6180 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:19.792610 6250 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 30. Preceding OpId from leader: term: 1 index: 31. (index mismatch)
I20251209 03:04:19.792800 6927 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:19.792917 6871 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:19.793059 6925 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:19.793759 6871 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:19.793915 6180 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:19.793915 6250 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:19.794796 6646 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.31: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:19.795500 5998 catalog_manager.cc:5654] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 30 to 31, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:19.796314 6099 consensus_queue.cc:237] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:19.796623 6647 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:19.796802 6922 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:19.797168 6250 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 1 index: 32. (index mismatch)
I20251209 03:04:19.797359 6927 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:19.797612 6871 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:19.797586 6647 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:19.797868 6250 raft_consensus.cc:2955] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Committing config change with OpId 1.32: config changed from index 31 to 32, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:19.798458 5999 catalog_manager.cc:5654] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 31 to 32, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New cstate: current_term: 1 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:19.798574 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 86b92235e7ad4a959951bdb816e39905 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:19.799065 6170 tablet_service.cc:1552] Processing DeleteTablet for tablet 86b92235e7ad4a959951bdb816e39905 with delete_type TABLET_DATA_TOMBSTONED (TS a6fcaebe8f1f4a319f00f9666d8ba592 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:19.799268 6931 tablet_replica.cc:333] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:19.799368 6931 raft_consensus.cc:2243] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:19.799458 6931 raft_consensus.cc:2272] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:19.799718 6931 ts_tablet_manager.cc:1916] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:19.801183 6931 ts_tablet_manager.cc:1929] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.31
I20251209 03:04:19.801242 6931 log.cc:1199] T 86b92235e7ad4a959951bdb816e39905 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/86b92235e7ad4a959951bdb816e39905
I20251209 03:04:19.801422 6571 raft_consensus.cc:3060] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 1 LEARNER]: Advancing to term 2
I20251209 03:04:19.801767 5987 catalog_manager.cc:4985] TS a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307): tablet 86b92235e7ad4a959951bdb816e39905 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:19.814990 6646 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEARNER]: Advancing to term 3
I20251209 03:04:19.815805 6646 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 3 LEARNER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251209 03:04:19.815997 6889 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:19.816767 6914 raft_consensus.cc:1064] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: attempting to promote NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e to VOTER
W20251209 03:04:19.816820 6914 raft_consensus.cc:1067] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: Unable to promote non-voter 03aecf9a33ec4b4ba130772d3c35d33e: Service unavailable: leader transfer in progress
I20251209 03:04:19.821134 6889 raft_consensus.cc:1064] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: attempting to promote NON_VOTER 37b7f916c7c843dfa479e106f762cfff to VOTER
I20251209 03:04:19.821338 6889 consensus_queue.cc:237] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 1.33, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:19.821867 6647 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 1 index: 34. (index mismatch)
I20251209 03:04:19.821888 6099 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 1 index: 34. (index mismatch)
I20251209 03:04:19.821966 6496 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 1 LEARNER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 33. Preceding OpId from leader: term: 1 index: 34. (index mismatch)
I20251209 03:04:19.822126 6914 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:19.822250 6889 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:19.822347 6934 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:19.822950 6889 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.34: config changed from index 33 to 34, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.823846 6496 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Committing config change with OpId 1.34: config changed from index 33 to 34, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.824823 6001 catalog_manager.cc:5654] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff reported cstate change: config changed from index 33 to 34, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.824853 6646 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.34: config changed from index 33 to 34, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.825078 6099 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Committing config change with OpId 1.34: config changed from index 33 to 34, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.826406 6180 consensus_queue.cc:237] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 1.34, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:19.826756 6646 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 34. Preceding OpId from leader: term: 1 index: 35. (index mismatch)
I20251209 03:04:19.826797 6496 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 34. Preceding OpId from leader: term: 1 index: 35. (index mismatch)
I20251209 03:04:19.826947 6934 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:19.827035 6914 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:19.827567 6934 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Committing config change with OpId 1.35: config changed from index 34 to 35, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.827685 6496 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Committing config change with OpId 1.35: config changed from index 34 to 35, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.827860 6646 raft_consensus.cc:2955] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Committing config change with OpId 1.35: config changed from index 34 to 35, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.828600 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet f71e8ee2ce1242608207622b267d96ae with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:19.828600 5997 catalog_manager.cc:5654] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 34 to 35, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New cstate: current_term: 1 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:19.829252 6086 tablet_service.cc:1552] Processing DeleteTablet for tablet f71e8ee2ce1242608207622b267d96ae with delete_type TABLET_DATA_TOMBSTONED (TS d36845214167428eb93883ecbe8f5385 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:19.829447 6940 tablet_replica.cc:333] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:19.829523 6940 raft_consensus.cc:2243] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:19.829713 6940 raft_consensus.cc:2272] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:19.829943 6940 ts_tablet_manager.cc:1916] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:19.831326 6940 ts_tablet_manager.cc:1929] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.34
I20251209 03:04:19.831382 6940 log.cc:1199] T f71e8ee2ce1242608207622b267d96ae P d36845214167428eb93883ecbe8f5385: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/f71e8ee2ce1242608207622b267d96ae
I20251209 03:04:19.831705 5987 catalog_manager.cc:4985] TS d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899): tablet f71e8ee2ce1242608207622b267d96ae (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:19.862912 6250 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251209 03:04:19.863109 6889 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:19.863824 6914 raft_consensus.cc:993] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:19.863898 6914 raft_consensus.cc:1081] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Signalling peer b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:19.864164 6250 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:35060
I20251209 03:04:19.864240 6250 raft_consensus.cc:493] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:19.864267 6250 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:19.864728 6099 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251209 03:04:19.864949 6936 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:19.865026 6250 raft_consensus.cc:515] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.864933 6889 raft_consensus.cc:1064] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: attempting to promote NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e to VOTER
W20251209 03:04:19.865140 6889 raft_consensus.cc:1067] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: Unable to promote non-voter 03aecf9a33ec4b4ba130772d3c35d33e: Service unavailable: leader transfer in progress
I20251209 03:04:19.865145 6250 leader_election.cc:290] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 4 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:19.865512 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:19.865569 6180 raft_consensus.cc:3055] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Stepping down as leader of term 3
I20251209 03:04:19.865592 6180 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:19.865662 6180 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.865749 6180 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:19.865811 6100 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:19.866413 6180 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 4.
I20251209 03:04:19.866540 6220 leader_election.cc:304] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters: d36845214167428eb93883ecbe8f5385
I20251209 03:04:19.866619 6904 raft_consensus.cc:2804] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Leader election won for term 4
I20251209 03:04:19.866675 6904 raft_consensus.cc:697] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:19.866729 6904 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:19.867127 5997 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 3 to 4, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 4 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 30 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:19.882174 6250 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:19.882398 6871 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:19.883075 6922 raft_consensus.cc:993] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: : Instructing follower b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:19.883147 6922 raft_consensus.cc:1081] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Signalling peer b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:19.883680 6180 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:19.883773 6252 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:35074
I20251209 03:04:19.883826 6252 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:19.883849 6252 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:19.883950 6922 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:19.884625 6252 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.884742 6252 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 5 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:19.884958 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:19.885017 6180 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:19.885752 6180 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 5.
I20251209 03:04:19.885834 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:19.885875 6220 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:19.885875 6099 raft_consensus.cc:3055] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Stepping down as leader of term 4
I20251209 03:04:19.885952 6099 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:19.885999 6904 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Leader election won for term 5
I20251209 03:04:19.886003 6099 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.886072 6904 raft_consensus.cc:697] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:19.886116 6099 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:19.886132 6904 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:19.886538 5997 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 4 to 5, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 5 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:19.886770 6099 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 5.
I20251209 03:04:19.888170 6496 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 3 LEARNER]: Advancing to term 4
I20251209 03:04:19.888856 6496 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 4 LEARNER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:20.021745 6252 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251209 03:04:20.022001 6922 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:20.023434 6180 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 4 index: 32. (index mismatch)
I20251209 03:04:20.023639 6922 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:20.247419 6898 tablet_copy_client.cc:538] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:20.248179 6898 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap starting.
I20251209 03:04:20.251542 6898 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=113 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:20.251710 6898 tablet_bootstrap.cc:492] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap complete.
I20251209 03:04:20.251804 6898 ts_tablet_manager.cc:1403] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251209 03:04:20.251931 6898 raft_consensus.cc:359] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:20.252010 6898 raft_consensus.cc:740] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2551ae093c914660b7f6e42bbea920fa, State: Initialized, Role: LEARNER
I20251209 03:04:20.252038 6898 consensus_queue.cc:260] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:20.252157 6898 ts_tablet_manager.cc:1434] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: Time spent starting tablet: real 0.000s user 0.002s sys 0.000s
I20251209 03:04:20.252343 6506 tablet_copy_service.cc:342] P 37b7f916c7c843dfa479e106f762cfff: Request end of tablet copy session 2551ae093c914660b7f6e42bbea920fa-a9a5e7d1a2de4913a09dfeb084d79ae1 received from {username='slave'} at 127.0.0.1:41540
I20251209 03:04:20.252411 6506 tablet_copy_service.cc:434] P 37b7f916c7c843dfa479e106f762cfff: ending tablet copy session 2551ae093c914660b7f6e42bbea920fa-a9a5e7d1a2de4913a09dfeb084d79ae1 on tablet a9a5e7d1a2de4913a09dfeb084d79ae1 with peer 2551ae093c914660b7f6e42bbea920fa
I20251209 03:04:20.298363 6099 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251209 03:04:20.298594 6913 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:20.299558 6496 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 4 LEARNER]: Advancing to term 5
I20251209 03:04:20.299674 6180 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251209 03:04:20.299890 6954 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:20.300393 6496 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 5 LEARNER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251209 03:04:20.300567 6954 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:20.301365 6099 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:20.301553 6913 raft_consensus.cc:1064] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: attempting to promote NON_VOTER 37b7f916c7c843dfa479e106f762cfff to VOTER
I20251209 03:04:20.301728 6913 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 5.34, Last appended by leader: 33, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:20.302093 6496 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 5 LEARNER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251209 03:04:20.302188 6099 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:20.302251 6954 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.302454 6958 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:20.302541 6099 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251209 03:04:20.302515 6180 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 5 index: 35. (index mismatch)
I20251209 03:04:20.302853 6958 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.303002 6954 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.303514 6175 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:20.303711 6904 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 LEADER]: Committing config change with OpId 5.35: config changed from index 32 to 35, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:20.303834 6646 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 3 LEARNER]: Advancing to term 4
I20251209 03:04:20.304459 6958 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:20.304548 6496 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 5 FOLLOWER]: Committing config change with OpId 5.35: config changed from index 32 to 35, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:20.304811 6175 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Committing config change with OpId 5.35: config changed from index 32 to 35, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:20.304442 6646 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 LEARNER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:20.305433 5999 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index 32 to 35, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:20.305578 6099 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Committing config change with OpId 5.35: config changed from index 32 to 35, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:20.306355 6252 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 33, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:20.306785 6904 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:20.307024 6496 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 5 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20251209 03:04:20.307047 6175 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20251209 03:04:20.307181 6958 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:20.307282 6954 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:20.307741 6954 raft_consensus.cc:1064] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: attempting to promote NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e to VOTER
I20251209 03:04:20.307883 6954 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 32, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:20.307900 6913 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 LEADER]: Committing config change with OpId 5.36: config changed from index 35 to 36, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:20.308157 6175 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Committing config change with OpId 5.36: config changed from index 35 to 36, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:20.308324 6180 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251209 03:04:20.308459 6958 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:20.308951 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 5ef8ec5789d64b8f931598cfcded95f2 with cas_config_opid_index 35: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:20.309062 5999 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index 35 to 36, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New cstate: current_term: 5 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:20.309130 6099 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251209 03:04:20.309103 6496 raft_consensus.cc:2955] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 5 FOLLOWER]: Committing config change with OpId 5.36: config changed from index 35 to 36, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:20.309307 6646 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 LEARNER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 4 index: 34. (index mismatch)
I20251209 03:04:20.309285 6958 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:20.309440 6958 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:20.309921 6913 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Committing config change with OpId 4.34: config changed from index 30 to 34, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.310698 6175 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 30 to 34, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.310748 6646 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 30 to 34, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.311703 5998 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 30 to 34, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.311800 6086 tablet_service.cc:1552] Processing DeleteTablet for tablet 5ef8ec5789d64b8f931598cfcded95f2 with delete_type TABLET_DATA_TOMBSTONED (TS d36845214167428eb93883ecbe8f5385 not found in new config with opid_index 36) from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:20.311770 6099 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Committing config change with OpId 4.34: config changed from index 30 to 34, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.311942 6940 tablet_replica.cc:333] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:20.312005 6940 raft_consensus.cc:2243] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:20.312052 6940 raft_consensus.cc:2272] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:20.312573 6940 ts_tablet_manager.cc:1916] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:20.312992 6252 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 32, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:20.313287 6646 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251209 03:04:20.313400 6099 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251209 03:04:20.313462 6959 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.313616 6958 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.313915 6940 ts_tablet_manager.cc:1929] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.35
I20251209 03:04:20.313966 6940 log.cc:1199] T 5ef8ec5789d64b8f931598cfcded95f2 P d36845214167428eb93883ecbe8f5385: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/5ef8ec5789d64b8f931598cfcded95f2
I20251209 03:04:20.314249 5987 catalog_manager.cc:4985] TS d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899): tablet 5ef8ec5789d64b8f931598cfcded95f2 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:20.314337 6099 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 34 to 35, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.314253 6958 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Committing config change with OpId 4.35: config changed from index 34 to 35, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.314503 6646 raft_consensus.cc:2955] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Committing config change with OpId 4.35: config changed from index 34 to 35, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.315403 6001 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 34 to 35, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New cstate: current_term: 4 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:20.316133 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 2b74f825019240c7950dcc767dd2e6a5 with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:20.316262 6170 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b74f825019240c7950dcc767dd2e6a5 with delete_type TABLET_DATA_TOMBSTONED (TS a6fcaebe8f1f4a319f00f9666d8ba592 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:20.316668 6960 tablet_replica.cc:333] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:20.316735 6960 raft_consensus.cc:2243] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:20.316808 6960 raft_consensus.cc:2272] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:20.317014 6960 ts_tablet_manager.cc:1916] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:20.318310 6960 ts_tablet_manager.cc:1929] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.34
I20251209 03:04:20.318367 6960 log.cc:1199] T 2b74f825019240c7950dcc767dd2e6a5 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/2b74f825019240c7950dcc767dd2e6a5
I20251209 03:04:20.318671 5987 catalog_manager.cc:4985] TS a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307): tablet 2b74f825019240c7950dcc767dd2e6a5 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:20.324487 6961 raft_consensus.cc:1064] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: attempting to promote NON_VOTER 2551ae093c914660b7f6e42bbea920fa to VOTER
I20251209 03:04:20.324744 6961 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 33, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:20.325213 6252 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:20.325233 6571 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:20.325345 6175 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:20.325467 6962 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.325618 6962 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.325826 6962 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.326325 6964 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 33 to 35, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.326422 6571 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 33 to 35, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.327209 5999 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 33 to 35, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:20.327255 6252 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 33 to 35, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.327737 6175 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 33 to 35, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.329155 6646 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 2.35, Last appended by leader: 33, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:20.329502 6571 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20251209 03:04:20.329537 6175 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20251209 03:04:20.329663 6866 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:20.329854 6866 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:20.330226 6961 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.330274 6571 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.330555 6175 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.331305 5997 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 35 to 36, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New cstate: current_term: 2 leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:20.331339 6048 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet cf139ecb17be4ec58d5c23e86eab88a4 with cas_config_opid_index 35: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:20.331998 6240 tablet_service.cc:1552] Processing DeleteTablet for tablet cf139ecb17be4ec58d5c23e86eab88a4 with delete_type TABLET_DATA_TOMBSTONED (TS b2fc3e20641e45a4b62732385fd4e036 not found in new config with opid_index 36) from {username='slave'} at 127.0.0.1:35054
I20251209 03:04:20.332192 6967 tablet_replica.cc:333] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:20.332262 6967 raft_consensus.cc:2243] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:20.332365 6967 raft_consensus.cc:2272] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:20.332613 6967 ts_tablet_manager.cc:1916] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:20.332971 6571 raft_consensus.cc:1217] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Deduplicated request from leader. Original: 1.32->[2.33-2.33] Dedup: 2.33->[]
I20251209 03:04:20.334026 6967 ts_tablet_manager.cc:1929] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.35
I20251209 03:04:20.334082 6967 log.cc:1199] T cf139ecb17be4ec58d5c23e86eab88a4 P b2fc3e20641e45a4b62732385fd4e036: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/cf139ecb17be4ec58d5c23e86eab88a4
I20251209 03:04:20.334370 5987 catalog_manager.cc:4985] TS b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597): tablet cf139ecb17be4ec58d5c23e86eab88a4 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:20.865582 6968 raft_consensus.cc:1064] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: attempting to promote NON_VOTER 2551ae093c914660b7f6e42bbea920fa to VOTER
I20251209 03:04:20.865818 6968 consensus_queue.cc:237] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 32, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:20.866258 6252 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:20.866300 6175 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:20.866318 6571 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:20.866431 6969 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:20.866605 6969 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:20.866751 6957 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:20.867601 6957 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 32 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.867749 6571 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.867811 6175 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.868747 6252 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.868841 6001 catalog_manager.cc:5654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa reported cstate change: config changed from index 32 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "37b7f916c7c843dfa479e106f762cfff" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.870381 6495 consensus_queue.cc:237] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:20.870729 6175 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:20.870725 6571 raft_consensus.cc:1275] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:20.870923 6968 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.871045 6971 consensus_queue.cc:1048] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:20.871562 6971 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.871773 6175 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.872538 6044 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:20.872754 5997 catalog_manager.cc:5654] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 34 to 35, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New cstate: current_term: 2 leader_uuid: "37b7f916c7c843dfa479e106f762cfff" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.872941 6571 raft_consensus.cc:2955] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:20.873819 6240 tablet_service.cc:1552] Processing DeleteTablet for tablet a9a5e7d1a2de4913a09dfeb084d79ae1 with delete_type TABLET_DATA_TOMBSTONED (TS b2fc3e20641e45a4b62732385fd4e036 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:35054
I20251209 03:04:20.874013 6978 tablet_replica.cc:333] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:20.874083 6978 raft_consensus.cc:2243] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:20.874167 6978 raft_consensus.cc:2272] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:20.874373 6978 ts_tablet_manager.cc:1916] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:20.875411 6978 ts_tablet_manager.cc:1929] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.34
I20251209 03:04:20.875455 6978 log.cc:1199] T a9a5e7d1a2de4913a09dfeb084d79ae1 P b2fc3e20641e45a4b62732385fd4e036: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/a9a5e7d1a2de4913a09dfeb084d79ae1
I20251209 03:04:20.875711 5987 catalog_manager.cc:4985] TS b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597): tablet a9a5e7d1a2de4913a09dfeb084d79ae1 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:21.257030 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:21.257606 6252 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "439ea3fec68444aeb8cc383b977a7f48"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "37b7f916c7c843dfa479e106f762cfff"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:21.257720 6252 raft_consensus.cc:606] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Received request to transfer leadership to TS 37b7f916c7c843dfa479e106f762cfff
I20251209 03:04:21.258036 6099 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
new_leader_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:58930
I20251209 03:04:21.258112 6099 raft_consensus.cc:606] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Received request to transfer leadership to TS b2fc3e20641e45a4b62732385fd4e036
I20251209 03:04:21.258433 6252 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "f9a52de6742a4e508e06374063561398"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:21.258505 6252 raft_consensus.cc:606] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:21.258649 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20251209 03:04:21.258720 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:21.417321 6959 raft_consensus.cc:993] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:21.417410 6959 raft_consensus.cc:1081] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:21.417618 6175 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "f9a52de6742a4e508e06374063561398"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43158
I20251209 03:04:21.417726 6175 raft_consensus.cc:493] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:21.417757 6175 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.418390 6175 raft_consensus.cc:515] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:21.418516 6175 leader_election.cc:290] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 2 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:21.418692 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 2 candidate_status { last_received { term: 1 index: 27 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:21.418759 6252 raft_consensus.cc:3055] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:21.418696 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 2 candidate_status { last_received { term: 1 index: 27 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:21.418784 6252 raft_consensus.cc:740] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:21.418797 6099 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.418836 6252 consensus_queue.cc:260] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:21.418913 6252 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.419360 6099 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 2.
I20251209 03:04:21.419425 6252 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 2.
I20251209 03:04:21.419539 6145 leader_election.cc:304] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:21.419627 6889 raft_consensus.cc:2804] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:21.419725 6889 raft_consensus.cc:697] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:21.419780 6889 consensus_queue.cc:237] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 27, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
I20251209 03:04:21.420279 5998 catalog_manager.cc:5654] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 1 to 2, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 2 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:21.614003 6959 raft_consensus.cc:993] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower 37b7f916c7c843dfa479e106f762cfff to start an election
I20251209 03:04:21.614089 6959 raft_consensus.cc:1081] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Signalling peer 37b7f916c7c843dfa479e106f762cfff to start an election
I20251209 03:04:21.614306 6495 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "439ea3fec68444aeb8cc383b977a7f48"
dest_uuid: "37b7f916c7c843dfa479e106f762cfff"
from {username='slave'} at 127.0.0.1:41510
I20251209 03:04:21.614384 6495 raft_consensus.cc:493] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:21.614404 6495 raft_consensus.cc:3060] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.615031 6495 raft_consensus.cc:515] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:21.615199 6495 leader_election.cc:290] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [CANDIDATE]: Term 2 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:21.615578 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "439ea3fec68444aeb8cc383b977a7f48" candidate_uuid: "37b7f916c7c843dfa479e106f762cfff" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:21.615670 6252 raft_consensus.cc:3055] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:21.615686 6252 raft_consensus.cc:740] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:21.615731 6252 consensus_queue.cc:260] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:21.615813 6252 raft_consensus.cc:3060] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.616407 6252 raft_consensus.cc:2468] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37b7f916c7c843dfa479e106f762cfff in term 2.
I20251209 03:04:21.616585 6461 leader_election.cc:304] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b7f916c7c843dfa479e106f762cfff, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:21.616664 6971 raft_consensus.cc:2804] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:21.616796 6971 raft_consensus.cc:697] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Becoming Leader. State: Replica: 37b7f916c7c843dfa479e106f762cfff, State: Running, Role: LEADER
I20251209 03:04:21.616853 6971 consensus_queue.cc:237] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:21.617610 5998 catalog_manager.cc:5654] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff reported cstate change: term changed from 1 to 2, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to 37b7f916c7c843dfa479e106f762cfff (127.2.170.196). New cstate: current_term: 2 leader_uuid: "37b7f916c7c843dfa479e106f762cfff" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:21.618264 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "439ea3fec68444aeb8cc383b977a7f48" candidate_uuid: "37b7f916c7c843dfa479e106f762cfff" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:21.618335 6099 raft_consensus.cc:3060] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.618963 6099 raft_consensus.cc:2468] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37b7f916c7c843dfa479e106f762cfff in term 2.
I20251209 03:04:21.676215 6922 raft_consensus.cc:993] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: : Instructing follower b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:21.676297 6922 raft_consensus.cc:1081] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Signalling peer b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:21.676466 6252 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:35074
I20251209 03:04:21.676541 6252 raft_consensus.cc:493] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:21.676563 6252 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.677119 6252 raft_consensus.cc:515] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:21.677254 6252 leader_election.cc:290] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 2 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151)
I20251209 03:04:21.677418 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:21.677440 6647 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 2 candidate_status { last_received { term: 1 index: 32 } } ignore_live_leader: true dest_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
I20251209 03:04:21.677505 6099 raft_consensus.cc:3055] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:21.677518 6647 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.677531 6099 raft_consensus.cc:740] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:21.677592 6099 consensus_queue.cc:260] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:21.677702 6099 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:21.678123 6647 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 2.
I20251209 03:04:21.678277 6099 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 2.
I20251209 03:04:21.678301 6221 leader_election.cc:304] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03aecf9a33ec4b4ba130772d3c35d33e, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:21.678375 6959 raft_consensus.cc:2804] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:21.678419 6959 raft_consensus.cc:697] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:21.678474 6959 consensus_queue.cc:237] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 1.32, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:21.678979 5998 catalog_manager.cc:5654] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 1 to 2, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 2 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20251209 03:04:21.873966 6049 auto_rebalancer.cc:253] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251209 03:04:21.874578 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 86b92235e7ad4a959951bdb816e39905, TS d36845214167428eb93883ecbe8f5385 -> TS 37b7f916c7c843dfa479e106f762cfff move failed, destination replica disappeared from tablet's Raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
W20251209 03:04:21.874634 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 86b92235e7ad4a959951bdb816e39905, TS d36845214167428eb93883ecbe8f5385 -> TS 37b7f916c7c843dfa479e106f762cfff move failed, destination replica disappeared from tablet's Raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
W20251209 03:04:21.874943 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 5ef8ec5789d64b8f931598cfcded95f2, TS a6fcaebe8f1f4a319f00f9666d8ba592 -> TS 2551ae093c914660b7f6e42bbea920fa move failed, destination replica disappeared from tablet's Raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
W20251209 03:04:21.874991 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 5ef8ec5789d64b8f931598cfcded95f2, TS a6fcaebe8f1f4a319f00f9666d8ba592 -> TS 2551ae093c914660b7f6e42bbea920fa move failed, destination replica disappeared from tablet's Raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
W20251209 03:04:21.875551 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet c5b028a080314d919052909cd10db307, TS b2fc3e20641e45a4b62732385fd4e036 -> TS 03aecf9a33ec4b4ba130772d3c35d33e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:21.875597 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet c5b028a080314d919052909cd10db307, TS b2fc3e20641e45a4b62732385fd4e036 -> TS 03aecf9a33ec4b4ba130772d3c35d33e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:21.875972 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet cf139ecb17be4ec58d5c23e86eab88a4, TS a6fcaebe8f1f4a319f00f9666d8ba592 -> TS 37b7f916c7c843dfa479e106f762cfff move failed, destination replica disappeared from tablet's Raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
W20251209 03:04:21.876005 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet cf139ecb17be4ec58d5c23e86eab88a4, TS a6fcaebe8f1f4a319f00f9666d8ba592 -> TS 37b7f916c7c843dfa479e106f762cfff move failed, destination replica disappeared from tablet's Raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
W20251209 03:04:21.876330 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 0e6b3f62d7894c69b4cf99bf1ad942ce, TS d36845214167428eb93883ecbe8f5385 -> TS 03aecf9a33ec4b4ba130772d3c35d33e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:21.876360 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 0e6b3f62d7894c69b4cf99bf1ad942ce, TS d36845214167428eb93883ecbe8f5385 -> TS 03aecf9a33ec4b4ba130772d3c35d33e move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } }
W20251209 03:04:21.876720 6049 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 439ea3fec68444aeb8cc383b977a7f48, TS b2fc3e20641e45a4b62732385fd4e036 -> TS 2551ae093c914660b7f6e42bbea920fa move failed, destination replica disappeared from tablet's Raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
W20251209 03:04:21.876753 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 439ea3fec68444aeb8cc383b977a7f48, TS b2fc3e20641e45a4b62732385fd4e036 -> TS 2551ae093c914660b7f6e42bbea920fa move failed, destination replica disappeared from tablet's Raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:21.960973 6252 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251209 03:04:21.961191 6889 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251209 03:04:21.962338 6099 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 2 index: 28. (index mismatch)
I20251209 03:04:21.962508 6889 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251209 03:04:22.103341 6252 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251209 03:04:22.103626 6971 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:22.104841 6099 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 1 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251209 03:04:22.105082 7000 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:22.121986 6647 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:22.122218 6959 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:22.123102 6099 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 1 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:22.123344 6959 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:22.884243 6571 consensus_queue.cc:237] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:22.884922 6099 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:22.884922 6252 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:22.885238 6881 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:22.886603 7012 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:22.887285 6881 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
W20251209 03:04:22.887692 6538 consensus_peers.cc:597] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa -> Peer 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151): Couldn't send request to peer 03aecf9a33ec4b4ba130772d3c35d33e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2beec1d9e1ff42cea8ff5a8e9024a6fe. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:22.887712 6099 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:22.888599 6252 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 32 to 34, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:22.888710 5998 catalog_manager.cc:5654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa reported cstate change: config changed from index 32 to 34, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New cstate: current_term: 2 leader_uuid: "2551ae093c914660b7f6e42bbea920fa" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:22.889278 6175 consensus_queue.cc:237] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 2.30, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:22.889812 6099 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
I20251209 03:04:22.889915 6252 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 30. Preceding OpId from leader: term: 2 index: 31. (index mismatch)
W20251209 03:04:22.890021 6146 consensus_peers.cc:597] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607): Couldn't send request to peer 2551ae093c914660b7f6e42bbea920fa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e6b3f62d7894c69b4cf99bf1ad942ce. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:22.890091 7006 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:22.890228 7021 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:22.890949 7021 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Committing config change with OpId 2.31: config changed from index -1 to 31, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:22.891070 6099 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index -1 to 31, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:22.891049 6252 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.31: config changed from index -1 to 31, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:22.892125 6001 catalog_manager.cc:5654] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index -1 to 31, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New cstate: current_term: 2 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:22.892227 6647 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 2.36, Last appended by leader: 33, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:22.892727 6571 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 36. Preceding OpId from leader: term: 2 index: 37. (index mismatch)
I20251209 03:04:22.892933 6175 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 36. Preceding OpId from leader: term: 2 index: 37. (index mismatch)
I20251209 03:04:22.893111 7026 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251209 03:04:22.893235 6979 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251209 03:04:22.893844 6979 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Committing config change with OpId 2.37: config changed from index 36 to 37, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:22.893987 6571 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.37: config changed from index 36 to 37, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:22.894937 5998 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 36 to 37, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New cstate: current_term: 2 leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:22.895211 6180 consensus_queue.cc:237] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 2.28, Last appended by leader: 27, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:22.895272 6175 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Committing config change with OpId 2.37: config changed from index 36 to 37, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:22.895632 6099 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251209 03:04:22.895632 6252 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 28. Preceding OpId from leader: term: 2 index: 29. (index mismatch)
I20251209 03:04:22.895805 7021 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
W20251209 03:04:22.895916 6143 consensus_peers.cc:597] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 -> Peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979): Couldn't send request to peer 37b7f916c7c843dfa479e106f762cfff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f9a52de6742a4e508e06374063561398. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:22.896046 6889 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 28, Time since last communication: 0.000s
I20251209 03:04:22.896559 7006 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Committing config change with OpId 2.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:22.896706 6252 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
W20251209 03:04:22.897437 6610 consensus_peers.cc:597] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e -> Peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979): Couldn't send request to peer 37b7f916c7c843dfa479e106f762cfff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf139ecb17be4ec58d5c23e86eab88a4. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:22.897578 6099 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Committing config change with OpId 2.29: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New config: { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:22.897742 6495 consensus_queue.cc:237] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:22.898172 6100 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
W20251209 03:04:22.898183 6462 consensus_peers.cc:597] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff -> Peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607): Couldn't send request to peer 2551ae093c914660b7f6e42bbea920fa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 439ea3fec68444aeb8cc383b977a7f48. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:22.898324 7000 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:22.897573 5999 catalog_manager.cc:5654] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index -1 to 29, NON_VOTER 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) added. New cstate: current_term: 2 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } } }
I20251209 03:04:22.898528 6252 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:22.898721 7000 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:22.898840 6988 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:22.899130 6099 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:22.899340 6252 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 31 to 33, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } } }
I20251209 03:04:22.899871 5998 catalog_manager.cc:5654] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff reported cstate change: config changed from index 31 to 33, NON_VOTER 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) added. New cstate: current_term: 2 leader_uuid: "37b7f916c7c843dfa479e106f762cfff" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:22.900441 6099 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 4.32, Last appended by leader: 31, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:22.900970 6175 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
W20251209 03:04:22.900990 6070 consensus_peers.cc:597] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 -> Peer 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151): Couldn't send request to peer 03aecf9a33ec4b4ba130772d3c35d33e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5b028a080314d919052909cd10db307. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:22.901131 6252 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 4 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:22.901212 7033 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:22.901319 7032 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:22.902067 7033 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Committing config change with OpId 4.33: config changed from index -1 to 33, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:22.902235 6252 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index -1 to 33, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:22.902246 6175 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Committing config change with OpId 4.33: config changed from index -1 to 33, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:22.903049 5997 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index -1 to 33, NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) added. New cstate: current_term: 4 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } } }
I20251209 03:04:23.258859 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:23.259298 6252 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:23.259382 6252 raft_consensus.cc:606] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Received request to transfer leadership to TS 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:23.259773 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "f9a52de6742a4e508e06374063561398"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:23.259846 6175 raft_consensus.cc:606] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Received request to transfer leadership to TS d36845214167428eb93883ecbe8f5385
I20251209 03:04:23.259967 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20251209 03:04:23.260071 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:23.370044 7036 ts_tablet_manager.cc:933] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: Initiating tablet copy from peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:23.370353 7036 tablet_copy_client.cc:323] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.196:43979
I20251209 03:04:23.370563 6506 tablet_copy_service.cc:140] P 37b7f916c7c843dfa479e106f762cfff: Received BeginTabletCopySession request for tablet 439ea3fec68444aeb8cc383b977a7f48 from peer 2551ae093c914660b7f6e42bbea920fa ({username='slave'} at 127.0.0.1:41540)
I20251209 03:04:23.370636 6506 tablet_copy_service.cc:161] P 37b7f916c7c843dfa479e106f762cfff: Beginning new tablet copy session on tablet 439ea3fec68444aeb8cc383b977a7f48 from peer 2551ae093c914660b7f6e42bbea920fa at {username='slave'} at 127.0.0.1:41540: session id = 2551ae093c914660b7f6e42bbea920fa-439ea3fec68444aeb8cc383b977a7f48
I20251209 03:04:23.371222 6506 tablet_copy_source_session.cc:215] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:23.371526 7036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 439ea3fec68444aeb8cc383b977a7f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:23.372987 7036 tablet_copy_client.cc:806] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:23.373093 7036 tablet_copy_client.cc:670] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:23.373615 7037 tablet_copy_client.cc:1025] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:23.373708 7007 raft_consensus.cc:993] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower 03aecf9a33ec4b4ba130772d3c35d33e to start an election
I20251209 03:04:23.373788 7038 raft_consensus.cc:1081] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Signalling peer 03aecf9a33ec4b4ba130772d3c35d33e to start an election
I20251209 03:04:23.373950 6647 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
from {username='slave'} at 127.0.0.1:40326
I20251209 03:04:23.374015 6647 raft_consensus.cc:493] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:23.374049 6647 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:23.374735 6647 raft_consensus.cc:515] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:23.374874 6647 leader_election.cc:290] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [CANDIDATE]: Term 5 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:23.374990 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" candidate_term: 5 candidate_status { last_received { term: 4 index: 35 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:23.375052 6252 raft_consensus.cc:3055] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Stepping down as leader of term 4
I20251209 03:04:23.375073 6252 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:23.375119 6252 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:23.375201 6252 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:23.375674 6252 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03aecf9a33ec4b4ba130772d3c35d33e in term 5.
I20251209 03:04:23.375769 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" candidate_term: 5 candidate_status { last_received { term: 4 index: 35 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:23.375811 6099 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:23.375844 6612 leader_election.cc:304] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03aecf9a33ec4b4ba130772d3c35d33e, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:23.375921 6979 raft_consensus.cc:2804] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 FOLLOWER]: Leader election won for term 5
I20251209 03:04:23.375970 6979 raft_consensus.cc:697] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 LEADER]: Becoming Leader. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Running, Role: LEADER
I20251209 03:04:23.376010 6979 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 4.35, Last appended by leader: 35, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:23.376350 6099 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03aecf9a33ec4b4ba130772d3c35d33e in term 5.
I20251209 03:04:23.376498 6001 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: term changed from 4 to 5, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198). New cstate: current_term: 5 leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:23.381021 7040 ts_tablet_manager.cc:933] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: Initiating tablet copy from peer 2551ae093c914660b7f6e42bbea920fa (127.2.170.197:41607)
I20251209 03:04:23.381228 7040 tablet_copy_client.cc:323] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.197:41607
I20251209 03:04:23.381412 6582 tablet_copy_service.cc:140] P 2551ae093c914660b7f6e42bbea920fa: Received BeginTabletCopySession request for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe from peer 03aecf9a33ec4b4ba130772d3c35d33e ({username='slave'} at 127.0.0.1:45376)
I20251209 03:04:23.381471 6582 tablet_copy_service.cc:161] P 2551ae093c914660b7f6e42bbea920fa: Beginning new tablet copy session on tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe from peer 03aecf9a33ec4b4ba130772d3c35d33e at {username='slave'} at 127.0.0.1:45376: session id = 03aecf9a33ec4b4ba130772d3c35d33e-2beec1d9e1ff42cea8ff5a8e9024a6fe
I20251209 03:04:23.382059 6582 tablet_copy_source_session.cc:215] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:23.382346 7040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:23.383564 7040 tablet_copy_client.cc:806] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:23.383709 7040 tablet_copy_client.cc:670] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:23.384527 7041 tablet_copy_client.cc:1025] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:23.431578 7042 ts_tablet_manager.cc:933] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: Initiating tablet copy from peer a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:23.431772 7042 tablet_copy_client.cc:323] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42307
I20251209 03:04:23.431936 6190 tablet_copy_service.cc:140] P a6fcaebe8f1f4a319f00f9666d8ba592: Received BeginTabletCopySession request for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce from peer 2551ae093c914660b7f6e42bbea920fa ({username='slave'} at 127.0.0.1:43212)
I20251209 03:04:23.432006 6190 tablet_copy_service.cc:161] P a6fcaebe8f1f4a319f00f9666d8ba592: Beginning new tablet copy session on tablet 0e6b3f62d7894c69b4cf99bf1ad942ce from peer 2551ae093c914660b7f6e42bbea920fa at {username='slave'} at 127.0.0.1:43212: session id = 2551ae093c914660b7f6e42bbea920fa-0e6b3f62d7894c69b4cf99bf1ad942ce
I20251209 03:04:23.432534 6190 tablet_copy_source_session.cc:215] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:23.432888 7042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:23.434337 7042 tablet_copy_client.cc:806] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:23.434485 7042 tablet_copy_client.cc:670] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:23.434985 7043 tablet_copy_client.cc:1025] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:23.452216 7044 ts_tablet_manager.cc:933] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: Initiating tablet copy from peer a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:23.452409 7044 tablet_copy_client.cc:323] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:42307
I20251209 03:04:23.452625 6190 tablet_copy_service.cc:140] P a6fcaebe8f1f4a319f00f9666d8ba592: Received BeginTabletCopySession request for tablet f9a52de6742a4e508e06374063561398 from peer 37b7f916c7c843dfa479e106f762cfff ({username='slave'} at 127.0.0.1:43190)
I20251209 03:04:23.452682 6190 tablet_copy_service.cc:161] P a6fcaebe8f1f4a319f00f9666d8ba592: Beginning new tablet copy session on tablet f9a52de6742a4e508e06374063561398 from peer 37b7f916c7c843dfa479e106f762cfff at {username='slave'} at 127.0.0.1:43190: session id = 37b7f916c7c843dfa479e106f762cfff-f9a52de6742a4e508e06374063561398
I20251209 03:04:23.453411 6190 tablet_copy_source_session.cc:215] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:23.453837 7044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a52de6742a4e508e06374063561398. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:23.455205 7044 tablet_copy_client.cc:806] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:23.455374 7044 tablet_copy_client.cc:670] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:23.455937 7045 tablet_copy_client.cc:1025] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:23.459955 7046 ts_tablet_manager.cc:933] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: Initiating tablet copy from peer 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151)
I20251209 03:04:23.460178 7046 tablet_copy_client.cc:323] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.198:34151
I20251209 03:04:23.463438 6657 tablet_copy_service.cc:140] P 03aecf9a33ec4b4ba130772d3c35d33e: Received BeginTabletCopySession request for tablet cf139ecb17be4ec58d5c23e86eab88a4 from peer 37b7f916c7c843dfa479e106f762cfff ({username='slave'} at 127.0.0.1:43620)
I20251209 03:04:23.463498 6657 tablet_copy_service.cc:161] P 03aecf9a33ec4b4ba130772d3c35d33e: Beginning new tablet copy session on tablet cf139ecb17be4ec58d5c23e86eab88a4 from peer 37b7f916c7c843dfa479e106f762cfff at {username='slave'} at 127.0.0.1:43620: session id = 37b7f916c7c843dfa479e106f762cfff-cf139ecb17be4ec58d5c23e86eab88a4
I20251209 03:04:23.464027 6657 tablet_copy_source_session.cc:215] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:23.464311 7046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf139ecb17be4ec58d5c23e86eab88a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:23.465503 7046 tablet_copy_client.cc:806] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:23.465641 7046 tablet_copy_client.cc:670] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:23.466249 7047 tablet_copy_client.cc:1025] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:23.473501 6889 raft_consensus.cc:993] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:23.473573 6889 raft_consensus.cc:1081] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Signalling peer d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:23.473734 6099 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "f9a52de6742a4e508e06374063561398"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:58880
I20251209 03:04:23.473806 6099 raft_consensus.cc:493] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:23.473835 6099 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:23.474505 6099 raft_consensus.cc:515] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:23.474623 6099 leader_election.cc:290] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 3 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:23.474789 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:23.474822 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 3 candidate_status { last_received { term: 2 index: 29 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:23.474869 6180 raft_consensus.cc:3055] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Stepping down as leader of term 2
I20251209 03:04:23.474910 6252 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:23.474915 6180 raft_consensus.cc:740] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:23.474989 6180 consensus_queue.cc:260] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:23.475083 6180 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:23.475484 6252 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 3.
I20251209 03:04:23.475615 6180 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 3.
I20251209 03:04:23.475662 6069 leader_election.cc:304] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:23.475750 7033 raft_consensus.cc:2804] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:04:23.475906 7033 raft_consensus.cc:697] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:23.475988 7033 consensus_queue.cc:237] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 29, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:23.476454 6001 catalog_manager.cc:5654] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 2 to 3, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 3 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:23.516661 7052 ts_tablet_manager.cc:933] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: Initiating tablet copy from peer d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899)
I20251209 03:04:23.516959 7052 tablet_copy_client.cc:323] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.193:43899
I20251209 03:04:23.517283 6114 tablet_copy_service.cc:140] P d36845214167428eb93883ecbe8f5385: Received BeginTabletCopySession request for tablet c5b028a080314d919052909cd10db307 from peer 03aecf9a33ec4b4ba130772d3c35d33e ({username='slave'} at 127.0.0.1:58938)
I20251209 03:04:23.517345 6114 tablet_copy_service.cc:161] P d36845214167428eb93883ecbe8f5385: Beginning new tablet copy session on tablet c5b028a080314d919052909cd10db307 from peer 03aecf9a33ec4b4ba130772d3c35d33e at {username='slave'} at 127.0.0.1:58938: session id = 03aecf9a33ec4b4ba130772d3c35d33e-c5b028a080314d919052909cd10db307
I20251209 03:04:23.517958 6114 tablet_copy_source_session.cc:215] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:23.518285 7052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b028a080314d919052909cd10db307. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:23.519534 7052 tablet_copy_client.cc:806] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:23.519650 7052 tablet_copy_client.cc:670] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:23.520311 7053 tablet_copy_client.cc:1025] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Injecting latency into file download: 2000
I20251209 03:04:23.853698 6099 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20251209 03:04:23.853946 6979 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:23.855464 6252 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 4 index: 35. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20251209 03:04:23.855681 6979 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:23.861647 6252 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
I20251209 03:04:23.862021 7033 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:23.863117 6180 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 2 index: 29. Preceding OpId from leader: term: 3 index: 30. (index mismatch)
W20251209 03:04:23.863125 6067 consensus_peers.cc:597] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 -> Peer 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979): Couldn't send request to peer 37b7f916c7c843dfa479e106f762cfff. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:23.863336 7051 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 30, Last known committed idx: 29, Time since last communication: 0.000s
I20251209 03:04:25.260236 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:25.260622 6100 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:58930
I20251209 03:04:25.260726 6100 raft_consensus.cc:606] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:25.260876 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:25.260937 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:25.373858 7036 tablet_copy_client.cc:538] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:25.374861 7036 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap starting.
I20251209 03:04:25.378998 7036 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:25.379212 7036 tablet_bootstrap.cc:492] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: Bootstrap complete.
I20251209 03:04:25.379334 7036 ts_tablet_manager.cc:1403] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20251209 03:04:25.379521 7036 raft_consensus.cc:359] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:25.379611 7036 raft_consensus.cc:740] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2551ae093c914660b7f6e42bbea920fa, State: Initialized, Role: LEARNER
I20251209 03:04:25.379643 7036 consensus_queue.cc:260] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:25.379765 7036 ts_tablet_manager.cc:1434] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.002s
I20251209 03:04:25.379945 6506 tablet_copy_service.cc:342] P 37b7f916c7c843dfa479e106f762cfff: Request end of tablet copy session 2551ae093c914660b7f6e42bbea920fa-439ea3fec68444aeb8cc383b977a7f48 received from {username='slave'} at 127.0.0.1:41540
I20251209 03:04:25.380003 6506 tablet_copy_service.cc:434] P 37b7f916c7c843dfa479e106f762cfff: ending tablet copy session 2551ae093c914660b7f6e42bbea920fa-439ea3fec68444aeb8cc383b977a7f48 on tablet 439ea3fec68444aeb8cc383b977a7f48 with peer 2551ae093c914660b7f6e42bbea920fa
I20251209 03:04:25.384747 7040 tablet_copy_client.cc:538] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:25.385473 7040 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap starting.
I20251209 03:04:25.389968 7040 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=34 ignored=0} inserts{seen=124 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:25.390188 7040 tablet_bootstrap.cc:492] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap complete.
I20251209 03:04:25.390311 7040 ts_tablet_manager.cc:1403] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20251209 03:04:25.390477 7040 raft_consensus.cc:359] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.390563 7040 raft_consensus.cc:740] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Initialized, Role: LEARNER
I20251209 03:04:25.390589 7040 consensus_queue.cc:260] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 34, Last appended: 2.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.390681 7040 ts_tablet_manager.cc:1434] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:25.390854 6582 tablet_copy_service.cc:342] P 2551ae093c914660b7f6e42bbea920fa: Request end of tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-2beec1d9e1ff42cea8ff5a8e9024a6fe received from {username='slave'} at 127.0.0.1:45376
I20251209 03:04:25.390923 6582 tablet_copy_service.cc:434] P 2551ae093c914660b7f6e42bbea920fa: ending tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-2beec1d9e1ff42cea8ff5a8e9024a6fe on tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe with peer 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:25.435262 7042 tablet_copy_client.cc:538] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:25.436286 7042 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: Bootstrap starting.
I20251209 03:04:25.440732 7042 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=134 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:25.440955 7042 tablet_bootstrap.cc:492] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: Bootstrap complete.
I20251209 03:04:25.441080 7042 ts_tablet_manager.cc:1403] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20251209 03:04:25.441270 7042 raft_consensus.cc:359] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:25.441360 7042 raft_consensus.cc:740] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2551ae093c914660b7f6e42bbea920fa, State: Initialized, Role: LEARNER
I20251209 03:04:25.441392 7042 consensus_queue.cc:260] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 2.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: NON_VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: true } }
I20251209 03:04:25.441498 7042 ts_tablet_manager.cc:1434] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:25.441723 6190 tablet_copy_service.cc:342] P a6fcaebe8f1f4a319f00f9666d8ba592: Request end of tablet copy session 2551ae093c914660b7f6e42bbea920fa-0e6b3f62d7894c69b4cf99bf1ad942ce received from {username='slave'} at 127.0.0.1:43212
I20251209 03:04:25.441780 6190 tablet_copy_service.cc:434] P a6fcaebe8f1f4a319f00f9666d8ba592: ending tablet copy session 2551ae093c914660b7f6e42bbea920fa-0e6b3f62d7894c69b4cf99bf1ad942ce on tablet 0e6b3f62d7894c69b4cf99bf1ad942ce with peer 2551ae093c914660b7f6e42bbea920fa
I20251209 03:04:25.456202 7044 tablet_copy_client.cc:538] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:25.457109 7044 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap starting.
I20251209 03:04:25.460711 7044 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=29 ignored=0} inserts{seen=112 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:25.460911 7044 tablet_bootstrap.cc:492] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap complete.
I20251209 03:04:25.461024 7044 ts_tablet_manager.cc:1403] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251209 03:04:25.461205 7044 raft_consensus.cc:359] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:25.461287 7044 raft_consensus.cc:740] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 37b7f916c7c843dfa479e106f762cfff, State: Initialized, Role: LEARNER
I20251209 03:04:25.461341 7044 consensus_queue.cc:260] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 29, Last appended: 2.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 29 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:25.461441 7044 ts_tablet_manager.cc:1434] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: Time spent starting tablet: real 0.000s user 0.000s sys 0.003s
I20251209 03:04:25.461778 6190 tablet_copy_service.cc:342] P a6fcaebe8f1f4a319f00f9666d8ba592: Request end of tablet copy session 37b7f916c7c843dfa479e106f762cfff-f9a52de6742a4e508e06374063561398 received from {username='slave'} at 127.0.0.1:43190
I20251209 03:04:25.461839 6190 tablet_copy_service.cc:434] P a6fcaebe8f1f4a319f00f9666d8ba592: ending tablet copy session 37b7f916c7c843dfa479e106f762cfff-f9a52de6742a4e508e06374063561398 on tablet f9a52de6742a4e508e06374063561398 with peer 37b7f916c7c843dfa479e106f762cfff
I20251209 03:04:25.466464 7046 tablet_copy_client.cc:538] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:25.467351 7046 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap starting.
I20251209 03:04:25.471544 7046 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap replayed 1/1 log segments. Stats: ops{read=37 overwritten=0 applied=37 ignored=0} inserts{seen=115 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:25.471728 7046 tablet_bootstrap.cc:492] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: Bootstrap complete.
I20251209 03:04:25.471836 7046 ts_tablet_manager.cc:1403] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251209 03:04:25.472005 7046 raft_consensus.cc:359] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:25.472074 7046 raft_consensus.cc:740] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 37b7f916c7c843dfa479e106f762cfff, State: Initialized, Role: LEARNER
I20251209 03:04:25.472102 7046 consensus_queue.cc:260] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 37, Last appended: 2.37, Last appended by leader: 37, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: true } }
I20251209 03:04:25.472416 7046 ts_tablet_manager.cc:1434] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251209 03:04:25.472586 6657 tablet_copy_service.cc:342] P 03aecf9a33ec4b4ba130772d3c35d33e: Request end of tablet copy session 37b7f916c7c843dfa479e106f762cfff-cf139ecb17be4ec58d5c23e86eab88a4 received from {username='slave'} at 127.0.0.1:43620
I20251209 03:04:25.472625 6657 tablet_copy_service.cc:434] P 03aecf9a33ec4b4ba130772d3c35d33e: ending tablet copy session 37b7f916c7c843dfa479e106f762cfff-cf139ecb17be4ec58d5c23e86eab88a4 on tablet cf139ecb17be4ec58d5c23e86eab88a4 with peer 37b7f916c7c843dfa479e106f762cfff
I20251209 03:04:25.481191 6571 raft_consensus.cc:1217] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Deduplicated request from leader. Original: 2.30->[2.31-2.31] Dedup: 2.31->[]
I20251209 03:04:25.503309 6647 raft_consensus.cc:1217] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.33->[2.34-2.34] Dedup: 2.34->[]
I20251209 03:04:25.505393 7051 raft_consensus.cc:993] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:25.505467 7051 raft_consensus.cc:1081] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:25.505611 6180 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43156
I20251209 03:04:25.505730 6180 raft_consensus.cc:493] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:25.505770 6180 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:25.506654 6180 raft_consensus.cc:515] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.506829 6180 leader_election.cc:290] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 5 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597)
I20251209 03:04:25.506987 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:25.506989 6100 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:25.507078 6252 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:25.507117 6100 raft_consensus.cc:3055] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Stepping down as leader of term 4
I20251209 03:04:25.507151 6100 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:25.507205 6100 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.507302 6100 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:25.507678 6252 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 5.
I20251209 03:04:25.507830 6145 leader_election.cc:304] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:25.507838 6100 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 5.
I20251209 03:04:25.507926 7068 raft_consensus.cc:2804] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Leader election won for term 5
I20251209 03:04:25.508020 7068 raft_consensus.cc:697] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:25.508078 7068 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.508603 6001 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 4 to 5, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 5 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:25.511000 6180 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:25.511070 6180 raft_consensus.cc:606] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 LEADER]: Received request to transfer leadership
I20251209 03:04:25.512707 6180 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:25.512769 6180 raft_consensus.cc:606] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 LEADER]: Received request to transfer leadership
W20251209 03:04:25.512892 5987 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for c5b028a080314d919052909cd10db307 already in progress [suppressed 1 similar messages]
I20251209 03:04:25.513201 6180 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:25.513248 6180 raft_consensus.cc:606] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 LEADER]: Received request to transfer leadership
W20251209 03:04:25.513413 6049 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for c5b028a080314d919052909cd10db307 already in progress
W20251209 03:04:25.513456 6049 auto_rebalancer.cc:263] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for c5b028a080314d919052909cd10db307 already in progress
I20251209 03:04:25.520530 7052 tablet_copy_client.cc:538] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:25.521531 7052 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap starting.
I20251209 03:04:25.524691 7052 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=130 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:25.524888 7052 tablet_bootstrap.cc:492] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: Bootstrap complete.
I20251209 03:04:25.524992 7052 ts_tablet_manager.cc:1403] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251209 03:04:25.525146 7052 raft_consensus.cc:359] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.525218 7052 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Initialized, Role: LEARNER
I20251209 03:04:25.525247 7052 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.525369 7052 ts_tablet_manager.cc:1434] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: Time spent starting tablet: real 0.000s user 0.000s sys 0.002s
I20251209 03:04:25.525743 6114 tablet_copy_service.cc:342] P d36845214167428eb93883ecbe8f5385: Request end of tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-c5b028a080314d919052909cd10db307 received from {username='slave'} at 127.0.0.1:58938
I20251209 03:04:25.525807 6114 tablet_copy_service.cc:434] P d36845214167428eb93883ecbe8f5385: ending tablet copy session 03aecf9a33ec4b4ba130772d3c35d33e-c5b028a080314d919052909cd10db307 on tablet c5b028a080314d919052909cd10db307 with peer 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:25.787806 6496 raft_consensus.cc:1217] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEARNER]: Deduplicated request from leader. Original: 2.36->[2.37-2.37] Dedup: 2.37->[]
I20251209 03:04:25.842447 6571 raft_consensus.cc:1217] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Deduplicated request from leader. Original: 2.32->[2.33-2.33] Dedup: 2.33->[]
I20251209 03:04:25.858820 6496 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEARNER]: Advancing to term 3
I20251209 03:04:25.885206 6252 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251209 03:04:25.885447 7068 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:25.886268 7068 raft_consensus.cc:993] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:25.886327 7068 raft_consensus.cc:1081] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 LEADER]: Signalling peer b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:25.886473 6250 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "c5b028a080314d919052909cd10db307"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:35060
I20251209 03:04:25.886543 6250 raft_consensus.cc:493] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:25.886564 6250 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:25.886770 6647 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 LEARNER]: Advancing to term 5
I20251209 03:04:25.886808 6100 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251209 03:04:25.886992 7076 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:25.887346 6250 raft_consensus.cc:515] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.887506 6250 leader_election.cc:290] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 6 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:25.887547 6647 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 LEARNER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251209 03:04:25.887704 6099 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 6 candidate_status { last_received { term: 5 index: 34 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:25.887805 6175 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5b028a080314d919052909cd10db307" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 6 candidate_status { last_received { term: 5 index: 34 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:25.887844 6175 raft_consensus.cc:3055] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 LEADER]: Stepping down as leader of term 5
I20251209 03:04:25.887863 6175 raft_consensus.cc:740] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:25.887944 6252 raft_consensus.cc:1240] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Rejecting Update request from peer a6fcaebe8f1f4a319f00f9666d8ba592 for earlier term 5. Current term is 6. Ops: []
I20251209 03:04:25.887908 6175 consensus_queue.cc:260] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 5.34, Last appended by leader: 34, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.887986 6175 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:25.888571 6175 raft_consensus.cc:2468] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 6.
I20251209 03:04:25.888835 6220 leader_election.cc:304] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, b2fc3e20641e45a4b62732385fd4e036; no voters: d36845214167428eb93883ecbe8f5385
I20251209 03:04:25.888922 7038 raft_consensus.cc:2804] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Leader election won for term 6
I20251209 03:04:25.889021 7038 raft_consensus.cc:697] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 6 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:25.889081 7038 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.34, Last appended by leader: 34, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } }
I20251209 03:04:25.889475 5998 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 5 to 6, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 6 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:25.971700 7088 raft_consensus.cc:1064] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592: attempting to promote NON_VOTER 2551ae093c914660b7f6e42bbea920fa to VOTER
I20251209 03:04:25.972034 7088 consensus_queue.cc:237] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 2.31, Last appended by leader: 29, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:25.972512 6099 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251209 03:04:25.972533 6571 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251209 03:04:25.972533 6252 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 2 index: 32. (index mismatch)
I20251209 03:04:25.972735 7093 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:25.972914 7088 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:25.973021 7076 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:25.973711 7068 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:25.973838 6252 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:25.974718 6099 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:25.974756 5998 catalog_manager.cc:5654] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 31 to 32, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:25.974828 6571 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.32: config changed from index 31 to 32, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:25.976111 6175 consensus_queue.cc:237] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 2.32, Last appended by leader: 29, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:25.976492 6099 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:25.976532 6571 raft_consensus.cc:1275] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 2 index: 32. Preceding OpId from leader: term: 2 index: 33. (index mismatch)
I20251209 03:04:25.976697 7068 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:25.976797 7088 consensus_queue.cc:1048] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:25.977268 7076 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:25.977356 6571 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:25.977383 6099 raft_consensus.cc:2955] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Committing config change with OpId 2.33: config changed from index 32 to 33, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:25.978277 5997 catalog_manager.cc:5654] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 32 to 33, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New cstate: current_term: 2 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 33 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:25.978650 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce with cas_config_opid_index 32: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:25.979125 6240 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e6b3f62d7894c69b4cf99bf1ad942ce with delete_type TABLET_DATA_TOMBSTONED (TS b2fc3e20641e45a4b62732385fd4e036 not found in new config with opid_index 33) from {username='slave'} at 127.0.0.1:35054
I20251209 03:04:25.979344 7097 tablet_replica.cc:333] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:25.979426 7097 raft_consensus.cc:2243] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:25.979521 7097 raft_consensus.cc:2272] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:25.979781 7097 ts_tablet_manager.cc:1916] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:25.981330 7097 ts_tablet_manager.cc:1929] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.32
I20251209 03:04:25.981406 7097 log.cc:1199] T 0e6b3f62d7894c69b4cf99bf1ad942ce P b2fc3e20641e45a4b62732385fd4e036: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/0e6b3f62d7894c69b4cf99bf1ad942ce
I20251209 03:04:25.981786 5987 catalog_manager.cc:4985] TS b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597): tablet 0e6b3f62d7894c69b4cf99bf1ad942ce (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:26.061336 7099 raft_consensus.cc:1064] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: attempting to promote NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e to VOTER
I20251209 03:04:26.061793 7099 consensus_queue.cc:237] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 32, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:26.062317 6252 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:26.062328 6100 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:26.062317 6647 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEARNER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:26.062566 7098 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:26.062672 7064 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:26.062949 7099 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:26.063683 7064 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.063781 6100 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.064584 6647 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.064704 6252 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.064733 6001 catalog_manager.cc:5654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 34 to 35, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2551ae093c914660b7f6e42bbea920fa" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.065658 6571 consensus_queue.cc:237] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 2.35, Last appended by leader: 32, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:26.066043 6252 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20251209 03:04:26.066044 6646 raft_consensus.cc:1275] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Refusing update from remote peer 2551ae093c914660b7f6e42bbea920fa: Log matching property violated. Preceding OpId in replica: term: 2 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20251209 03:04:26.066298 7099 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:26.066498 7098 consensus_queue.cc:1048] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:26.066962 7098 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 2 LEADER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.067070 6646 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.067083 6252 raft_consensus.cc:2955] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.36: config changed from index 35 to 36, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.067858 6048 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe with cas_config_opid_index 35: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:26.068055 6001 catalog_manager.cc:5654] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 35 to 36, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New cstate: current_term: 2 leader_uuid: "2551ae093c914660b7f6e42bbea920fa" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.068750 6086 tablet_service.cc:1552] Processing DeleteTablet for tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe with delete_type TABLET_DATA_TOMBSTONED (TS d36845214167428eb93883ecbe8f5385 not found in new config with opid_index 36) from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:26.068974 7104 tablet_replica.cc:333] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:26.069072 7104 raft_consensus.cc:2243] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:26.069181 7104 raft_consensus.cc:2272] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:26.069387 7104 ts_tablet_manager.cc:1916] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:26.071148 7104 ts_tablet_manager.cc:1929] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.35
I20251209 03:04:26.071209 7104 log.cc:1199] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P d36845214167428eb93883ecbe8f5385: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/2beec1d9e1ff42cea8ff5a8e9024a6fe
I20251209 03:04:26.071667 5987 catalog_manager.cc:4985] TS d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899): tablet 2beec1d9e1ff42cea8ff5a8e9024a6fe (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:26.253304 7105 raft_consensus.cc:1064] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: attempting to promote NON_VOTER 2551ae093c914660b7f6e42bbea920fa to VOTER
I20251209 03:04:26.253638 7105 consensus_queue.cc:237] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 31, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:26.254151 6571 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEARNER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:26.254235 6252 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:26.254273 6099 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 2 index: 34. (index mismatch)
I20251209 03:04:26.254336 7108 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:26.254470 7105 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:26.254568 7105 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:26.255309 7108 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.255405 6099 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.255546 6571 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.256554 6252 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.34: config changed from index 33 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New config: { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.256552 5999 catalog_manager.cc:5654] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa reported cstate change: config changed from index 33 to 34, 2551ae093c914660b7f6e42bbea920fa (127.2.170.197) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "37b7f916c7c843dfa479e106f762cfff" committed_config { opid_index: 34 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.257524 6496 consensus_queue.cc:237] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 2.34, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:26.257949 6571 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:26.258049 6252 raft_consensus.cc:1275] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Refusing update from remote peer 37b7f916c7c843dfa479e106f762cfff: Log matching property violated. Preceding OpId in replica: term: 2 index: 34. Preceding OpId from leader: term: 2 index: 35. (index mismatch)
I20251209 03:04:26.258167 7106 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:26.258265 7108 consensus_queue.cc:1048] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:26.258849 7106 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.258955 6252 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.259007 6571 raft_consensus.cc:2955] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.35: config changed from index 34 to 35, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.259873 5997 catalog_manager.cc:5654] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa reported cstate change: config changed from index 34 to 35, VOTER d36845214167428eb93883ecbe8f5385 (127.2.170.193) evicted. New cstate: current_term: 2 leader_uuid: "37b7f916c7c843dfa479e106f762cfff" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } }
I20251209 03:04:26.260197 6044 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 439ea3fec68444aeb8cc383b977a7f48 with cas_config_opid_index 34: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:26.260571 6086 tablet_service.cc:1552] Processing DeleteTablet for tablet 439ea3fec68444aeb8cc383b977a7f48 with delete_type TABLET_DATA_TOMBSTONED (TS d36845214167428eb93883ecbe8f5385 not found in new config with opid_index 35) from {username='slave'} at 127.0.0.1:58876
I20251209 03:04:26.260697 7104 tablet_replica.cc:333] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:26.260773 7104 raft_consensus.cc:2243] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:26.260860 7104 raft_consensus.cc:2272] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:26.261089 7104 ts_tablet_manager.cc:1916] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:26.262445 7104 ts_tablet_manager.cc:1929] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.34
I20251209 03:04:26.262503 7104 log.cc:1199] T 439ea3fec68444aeb8cc383b977a7f48 P d36845214167428eb93883ecbe8f5385: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/439ea3fec68444aeb8cc383b977a7f48
I20251209 03:04:26.262921 5987 catalog_manager.cc:4985] TS d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899): tablet 439ea3fec68444aeb8cc383b977a7f48 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:26.362741 7065 raft_consensus.cc:1064] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: attempting to promote NON_VOTER 37b7f916c7c843dfa479e106f762cfff to VOTER
I20251209 03:04:26.362969 7065 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 2.37, Last appended by leader: 33, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:26.363552 6571 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 37. Preceding OpId from leader: term: 2 index: 38. (index mismatch)
I20251209 03:04:26.363567 6496 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEARNER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 37. Preceding OpId from leader: term: 2 index: 38. (index mismatch)
I20251209 03:04:26.363552 6175 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 37. Preceding OpId from leader: term: 2 index: 38. (index mismatch)
I20251209 03:04:26.363826 7114 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251209 03:04:26.363965 7115 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251209 03:04:26.364264 7112 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251209 03:04:26.364940 7115 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Committing config change with OpId 2.38: config changed from index 37 to 38, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.365042 6175 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Committing config change with OpId 2.38: config changed from index 37 to 38, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.365067 6496 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Committing config change with OpId 2.38: config changed from index 37 to 38, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.366238 6571 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.38: config changed from index 37 to 38, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.366816 6001 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff reported cstate change: config changed from index 37 to 38, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" committed_config { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.367755 6647 consensus_queue.cc:237] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 2.38, Last appended by leader: 33, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:26.368129 6571 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 38. Preceding OpId from leader: term: 2 index: 39. (index mismatch)
I20251209 03:04:26.368142 6496 raft_consensus.cc:1275] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 2 index: 38. Preceding OpId from leader: term: 2 index: 39. (index mismatch)
I20251209 03:04:26.368335 7065 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20251209 03:04:26.368453 7115 consensus_queue.cc:1048] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20251209 03:04:26.368873 7115 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Committing config change with OpId 2.39: config changed from index 38 to 39, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.368994 6571 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Committing config change with OpId 2.39: config changed from index 38 to 39, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.369863 6048 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet cf139ecb17be4ec58d5c23e86eab88a4 with cas_config_opid_index 38: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:26.369891 5997 catalog_manager.cc:5654] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: config changed from index 38 to 39, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New cstate: current_term: 2 leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:26.369956 6496 raft_consensus.cc:2955] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Committing config change with OpId 2.39: config changed from index 38 to 39, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.371038 6170 tablet_service.cc:1552] Processing DeleteTablet for tablet cf139ecb17be4ec58d5c23e86eab88a4 with delete_type TABLET_DATA_TOMBSTONED (TS a6fcaebe8f1f4a319f00f9666d8ba592 not found in new config with opid_index 39) from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:26.371271 7119 tablet_replica.cc:333] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:26.371343 7119 raft_consensus.cc:2243] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:26.371476 7119 raft_consensus.cc:2272] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:26.371723 7119 ts_tablet_manager.cc:1916] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:26.373035 7119 ts_tablet_manager.cc:1929] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.38
I20251209 03:04:26.373093 7119 log.cc:1199] T cf139ecb17be4ec58d5c23e86eab88a4 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/cf139ecb17be4ec58d5c23e86eab88a4
I20251209 03:04:26.373392 5987 catalog_manager.cc:4985] TS a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307): tablet cf139ecb17be4ec58d5c23e86eab88a4 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:26.411972 7051 raft_consensus.cc:1064] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385: attempting to promote NON_VOTER 37b7f916c7c843dfa479e106f762cfff to VOTER
I20251209 03:04:26.412202 7051 consensus_queue.cc:237] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 30, Committed index: 30, Last appended: 3.30, Last appended by leader: 29, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:26.412746 6496 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 3 LEARNER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251209 03:04:26.412797 6252 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251209 03:04:26.412793 6175 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 30. Preceding OpId from leader: term: 3 index: 31. (index mismatch)
I20251209 03:04:26.413023 7051 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:26.413156 7120 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:26.413405 7123 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 31, Last known committed idx: 30, Time since last communication: 0.000s
I20251209 03:04:26.414170 6252 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 29 to 31, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.414052 7120 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Committing config change with OpId 3.31: config changed from index 29 to 31, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.414274 6175 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 29 to 31, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.415162 5998 catalog_manager.cc:5654] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: config changed from index 29 to 31, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.415179 6496 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 3 FOLLOWER]: Committing config change with OpId 3.31: config changed from index 29 to 31, 37b7f916c7c843dfa479e106f762cfff (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } attrs { replace: true } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.416580 6099 consensus_queue.cc:237] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 3.31, Last appended by leader: 29, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:26.416940 6496 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 3 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251209 03:04:26.417109 6175 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 3 index: 31. Preceding OpId from leader: term: 3 index: 32. (index mismatch)
I20251209 03:04:26.417140 7122 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:26.417490 7122 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251209 03:04:26.417642 7123 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Committing config change with OpId 3.32: config changed from index 31 to 32, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.417774 6496 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 31 to 32, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.418190 6175 raft_consensus.cc:2955] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Committing config change with OpId 3.32: config changed from index 31 to 32, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New config: { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } } }
I20251209 03:04:26.418726 5999 catalog_manager.cc:5654] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 31 to 32, VOTER b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) evicted. New cstate: current_term: 3 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:04:26.418802 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet f9a52de6742a4e508e06374063561398 with cas_config_opid_index 31: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:26.419557 6240 tablet_service.cc:1552] Processing DeleteTablet for tablet f9a52de6742a4e508e06374063561398 with delete_type TABLET_DATA_TOMBSTONED (TS b2fc3e20641e45a4b62732385fd4e036 not found in new config with opid_index 32) from {username='slave'} at 127.0.0.1:35054
I20251209 03:04:26.419695 7097 tablet_replica.cc:333] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:26.419785 7097 raft_consensus.cc:2243] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:26.419843 7097 raft_consensus.cc:2272] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:26.420085 7097 ts_tablet_manager.cc:1916] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:26.421314 7097 ts_tablet_manager.cc:1929] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.31
I20251209 03:04:26.421365 7097 log.cc:1199] T f9a52de6742a4e508e06374063561398 P b2fc3e20641e45a4b62732385fd4e036: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/f9a52de6742a4e508e06374063561398
I20251209 03:04:26.421726 5987 catalog_manager.cc:4985] TS b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597): tablet f9a52de6742a4e508e06374063561398 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:26.421838 6100 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:26.422350 6100 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20251209 03:04:26.422560 7038 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:26.423733 6175 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 5 index: 34. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20251209 03:04:26.423735 6647 raft_consensus.cc:3060] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 LEARNER]: Advancing to term 6
I20251209 03:04:26.423941 7128 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:26.424468 6647 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 LEARNER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20251209 03:04:26.424722 7128 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: NON_VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:26.425557 7130 raft_consensus.cc:1064] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036: attempting to promote NON_VOTER 03aecf9a33ec4b4ba130772d3c35d33e to VOTER
I20251209 03:04:26.425765 7130 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 6.35, Last appended by leader: 34, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:26.426144 6647 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 LEARNER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 6 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20251209 03:04:26.426172 6175 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 6 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20251209 03:04:26.426218 6100 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 6 index: 35. Preceding OpId from leader: term: 6 index: 36. (index mismatch)
I20251209 03:04:26.426388 7130 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:26.426563 7126 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:26.426700 7038 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:26.427297 6175 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Committing config change with OpId 6.36: config changed from index 33 to 36, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.427327 6100 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Committing config change with OpId 6.36: config changed from index 33 to 36, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.427198 7130 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 6 LEADER]: Committing config change with OpId 6.36: config changed from index 33 to 36, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.428411 6001 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: config changed from index 33 to 36, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.428903 6647 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Committing config change with OpId 6.36: config changed from index 33 to 36, 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) changed from NON_VOTER to VOTER. New config: { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } attrs { replace: true } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.429845 6252 consensus_queue.cc:237] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 6.36, Last appended by leader: 34, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:26.430248 6647 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 6 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251209 03:04:26.430331 6100 raft_consensus.cc:1275] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 6 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251209 03:04:26.430502 7126 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251209 03:04:26.430584 7128 consensus_queue.cc:1048] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251209 03:04:26.431190 7134 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 6 LEADER]: Committing config change with OpId 6.37: config changed from index 36 to 37, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.431260 6100 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Committing config change with OpId 6.37: config changed from index 36 to 37, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.431278 6647 raft_consensus.cc:2955] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Committing config change with OpId 6.37: config changed from index 36 to 37, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.432034 5987 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet c5b028a080314d919052909cd10db307 with cas_config_opid_index 36: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:26.432168 5997 catalog_manager.cc:5654] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 reported cstate change: config changed from index 36 to 37, VOTER a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) evicted. New cstate: current_term: 6 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } }
I20251209 03:04:26.432881 6170 tablet_service.cc:1552] Processing DeleteTablet for tablet c5b028a080314d919052909cd10db307 with delete_type TABLET_DATA_TOMBSTONED (TS a6fcaebe8f1f4a319f00f9666d8ba592 not found in new config with opid_index 37) from {username='slave'} at 127.0.0.1:43150
I20251209 03:04:26.432994 7119 tablet_replica.cc:333] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:26.433063 7119 raft_consensus.cc:2243] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:26.433121 7119 raft_consensus.cc:2272] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:26.433477 7119 ts_tablet_manager.cc:1916] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:26.434692 7119 ts_tablet_manager.cc:1929] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.36
I20251209 03:04:26.434739 7119 log.cc:1199] T c5b028a080314d919052909cd10db307 P a6fcaebe8f1f4a319f00f9666d8ba592: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/c5b028a080314d919052909cd10db307
I20251209 03:04:26.435007 5987 catalog_manager.cc:4985] TS a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307): tablet c5b028a080314d919052909cd10db307 (table test-workload [id=f83c37083814476fb11fdb400ba48a2a]) successfully deleted
I20251209 03:04:27.261093 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:27.261629 6252 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:27.261770 6252 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:27.261976 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:27.262043 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:27.269743 7130 raft_consensus.cc:993] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:27.269821 7130 raft_consensus.cc:1081] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:27.270017 6175 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43158
I20251209 03:04:27.270116 6175 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:27.270139 6175 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:27.270725 6175 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:27.270876 6175 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 6 election: Requested vote from peers b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597), 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:27.271009 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 6 candidate_status { last_received { term: 5 index: 36 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:27.271040 6496 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 6 candidate_status { last_received { term: 5 index: 36 } } ignore_live_leader: true dest_uuid: "37b7f916c7c843dfa479e106f762cfff"
I20251209 03:04:27.271123 6496 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:27.271107 6252 raft_consensus.cc:3055] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 LEADER]: Stepping down as leader of term 5
I20251209 03:04:27.271176 6252 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:27.271239 6252 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:27.271322 6252 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:27.271770 6496 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 6.
I20251209 03:04:27.271883 6252 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 6.
I20251209 03:04:27.271950 6143 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b7f916c7c843dfa479e106f762cfff, a6fcaebe8f1f4a319f00f9666d8ba592; no voters:
I20251209 03:04:27.272039 7068 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Leader election won for term 6
I20251209 03:04:27.272149 7068 raft_consensus.cc:697] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:27.272197 7068 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:27.272735 5999 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 5 to 6, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 6 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:27.667279 6252 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251209 03:04:27.667579 7068 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251209 03:04:27.668881 6496 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 6 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251209 03:04:27.669055 7144 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251209 03:04:29.262233 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:29.262759 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:29.262858 6175 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 LEADER]: Received request to transfer leadership to TS b2fc3e20641e45a4b62732385fd4e036
I20251209 03:04:29.263090 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:29.263221 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:29.421861 7144 raft_consensus.cc:993] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:29.421967 7144 raft_consensus.cc:1081] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 LEADER]: Signalling peer b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:29.422178 6252 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:35060
I20251209 03:04:29.422267 6252 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:29.422290 6252 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Advancing to term 7
I20251209 03:04:29.422900 6252 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:29.423051 6252 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 7 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:29.423277 6175 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 7 candidate_status { last_received { term: 6 index: 37 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:29.423298 6496 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 7 candidate_status { last_received { term: 6 index: 37 } } ignore_live_leader: true dest_uuid: "37b7f916c7c843dfa479e106f762cfff"
I20251209 03:04:29.423368 6175 raft_consensus.cc:3055] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 LEADER]: Stepping down as leader of term 6
I20251209 03:04:29.423379 6496 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 6 FOLLOWER]: Advancing to term 7
I20251209 03:04:29.423391 6175 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:29.423446 6175 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 6.37, Last appended by leader: 37, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:29.423535 6175 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 6 FOLLOWER]: Advancing to term 7
I20251209 03:04:29.423992 6496 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 7.
I20251209 03:04:29.424136 6175 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 7.
I20251209 03:04:29.424196 6218 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b7f916c7c843dfa479e106f762cfff, b2fc3e20641e45a4b62732385fd4e036; no voters:
I20251209 03:04:29.424285 7130 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 7 FOLLOWER]: Leader election won for term 7
I20251209 03:04:29.424381 7130 raft_consensus.cc:697] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 7 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:29.424443 7130 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 6.37, Last appended by leader: 37, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:29.425009 5999 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 6 to 7, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 7 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:29.917214 6496 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 7 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 6 index: 37. Preceding OpId from leader: term: 7 index: 38. (index mismatch)
I20251209 03:04:29.917564 7130 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251209 03:04:29.918977 6175 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 7 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 6 index: 37. Preceding OpId from leader: term: 7 index: 38. (index mismatch)
I20251209 03:04:29.919246 7150 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20251209 03:04:31.263397 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:31.263947 6252 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:31.264045 6252 raft_consensus.cc:606] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Received request to transfer leadership to TS d36845214167428eb93883ecbe8f5385
I20251209 03:04:31.264288 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:31.264381 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:31.569352 7162 raft_consensus.cc:993] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:31.569435 7162 raft_consensus.cc:1081] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Signalling peer d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:31.569639 6100 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:58884
I20251209 03:04:31.569731 6100 raft_consensus.cc:493] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:31.569762 6100 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:31.570411 6100 raft_consensus.cc:515] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:31.570555 6100 leader_election.cc:290] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 3 election: Requested vote from peers b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597), 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151)
I20251209 03:04:31.570704 6647 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: true dest_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
I20251209 03:04:31.570772 6647 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:31.570711 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:31.570811 6252 raft_consensus.cc:3055] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Stepping down as leader of term 2
I20251209 03:04:31.570845 6252 raft_consensus.cc:740] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:31.570906 6252 consensus_queue.cc:260] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:31.570997 6252 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:04:31.571354 6647 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 3.
I20251209 03:04:31.571524 6070 leader_election.cc:304] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03aecf9a33ec4b4ba130772d3c35d33e, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:31.571606 7123 raft_consensus.cc:2804] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:04:31.571592 6252 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 3.
I20251209 03:04:31.571733 7123 raft_consensus.cc:697] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:31.571795 7123 consensus_queue.cc:237] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:31.572305 5999 catalog_manager.cc:5654] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 2 to 3, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 3 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:32.129105 6252 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20251209 03:04:32.129411 7123 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:32.130575 6647 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 3 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
I20251209 03:04:32.130899 7172 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:33.264575 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:33.265233 6100 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "f9a52de6742a4e508e06374063561398"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:58930
I20251209 03:04:33.265350 6100 raft_consensus.cc:606] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:33.265559 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:33.265628 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:33.545082 7179 raft_consensus.cc:993] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:33.545167 7179 raft_consensus.cc:1081] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:33.545428 6175 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "f9a52de6742a4e508e06374063561398"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43156
I20251209 03:04:33.545507 6175 raft_consensus.cc:493] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:33.545537 6175 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:33.546237 6175 raft_consensus.cc:515] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:33.546386 6175 leader_election.cc:290] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 4 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:33.546540 6100 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:33.546639 6496 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 4 candidate_status { last_received { term: 3 index: 32 } } ignore_live_leader: true dest_uuid: "37b7f916c7c843dfa479e106f762cfff"
I20251209 03:04:33.546779 6100 raft_consensus.cc:3055] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Stepping down as leader of term 3
I20251209 03:04:33.546782 6496 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:33.546813 6100 raft_consensus.cc:740] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:33.546878 6100 consensus_queue.cc:260] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:33.546967 6100 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:33.547627 6100 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 4.
I20251209 03:04:33.547627 6496 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 4.
I20251209 03:04:33.547849 6145 leader_election.cc:304] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6fcaebe8f1f4a319f00f9666d8ba592, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:33.547958 7144 raft_consensus.cc:2804] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Leader election won for term 4
I20251209 03:04:33.548074 7144 raft_consensus.cc:697] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:33.548146 7144 consensus_queue.cc:237] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 32, Committed index: 32, Last appended: 3.32, Last appended by leader: 32, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:33.548700 5999 catalog_manager.cc:5654] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 3 to 4, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 4 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:33.940761 6100 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:33.941020 7144 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:33.942399 6496 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 4 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 3 index: 32. Preceding OpId from leader: term: 4 index: 33. (index mismatch)
I20251209 03:04:33.942581 7144 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 33, Last known committed idx: 32, Time since last communication: 0.000s
I20251209 03:04:35.265829 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:35.266408 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "f71e8ee2ce1242608207622b267d96ae"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:35.266505 6175 raft_consensus.cc:606] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Received request to transfer leadership to TS 03aecf9a33ec4b4ba130772d3c35d33e
I20251209 03:04:35.266719 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:35.266782 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:35.387696 7144 raft_consensus.cc:993] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower 03aecf9a33ec4b4ba130772d3c35d33e to start an election
I20251209 03:04:35.387775 7144 raft_consensus.cc:1081] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Signalling peer 03aecf9a33ec4b4ba130772d3c35d33e to start an election
I20251209 03:04:35.387959 6647 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "f71e8ee2ce1242608207622b267d96ae"
dest_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
from {username='slave'} at 127.0.0.1:40306
I20251209 03:04:35.388048 6647 raft_consensus.cc:493] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:35.388069 6647 raft_consensus.cc:3060] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:35.388676 6647 raft_consensus.cc:515] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:35.388833 6647 leader_election.cc:290] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [CANDIDATE]: Term 2 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:35.388991 6496 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f71e8ee2ce1242608207622b267d96ae" candidate_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" candidate_term: 2 candidate_status { last_received { term: 1 index: 35 } } ignore_live_leader: true dest_uuid: "37b7f916c7c843dfa479e106f762cfff"
I20251209 03:04:35.389065 6496 raft_consensus.cc:3060] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:35.388996 6175 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f71e8ee2ce1242608207622b267d96ae" candidate_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" candidate_term: 2 candidate_status { last_received { term: 1 index: 35 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:35.389114 6175 raft_consensus.cc:3055] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:35.389149 6175 raft_consensus.cc:740] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:35.389222 6175 consensus_queue.cc:260] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 1.35, Last appended by leader: 35, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:35.389303 6175 raft_consensus.cc:3060] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:35.389750 6496 raft_consensus.cc:2468] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03aecf9a33ec4b4ba130772d3c35d33e in term 2.
I20251209 03:04:35.389928 6610 leader_election.cc:304] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03aecf9a33ec4b4ba130772d3c35d33e, 37b7f916c7c843dfa479e106f762cfff; no voters:
I20251209 03:04:35.390054 7114 raft_consensus.cc:2804] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:35.390108 6175 raft_consensus.cc:2468] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03aecf9a33ec4b4ba130772d3c35d33e in term 2.
I20251209 03:04:35.390192 7114 raft_consensus.cc:697] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Becoming Leader. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Running, Role: LEADER
I20251209 03:04:35.390264 7114 consensus_queue.cc:237] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 1.35, Last appended by leader: 35, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:35.390791 6001 catalog_manager.cc:5654] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e reported cstate change: term changed from 1 to 2, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198). New cstate: current_term: 2 leader_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:35.934346 6175 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20251209 03:04:35.934669 7114 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:35.936123 6496 raft_consensus.cc:1275] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Refusing update from remote peer 03aecf9a33ec4b4ba130772d3c35d33e: Log matching property violated. Preceding OpId in replica: term: 1 index: 35. Preceding OpId from leader: term: 2 index: 36. (index mismatch)
I20251209 03:04:35.936383 7114 consensus_queue.cc:1048] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.000s
I20251209 03:04:37.267027 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:37.270248 6647 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
mode: GRACEFUL
new_leader_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:43732
I20251209 03:04:37.270347 6647 raft_consensus.cc:606] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 LEADER]: Received request to transfer leadership to TS d36845214167428eb93883ecbe8f5385
I20251209 03:04:37.270537 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:37.270605 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:37.392254 7207 raft_consensus.cc:993] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: : Instructing follower d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:37.392340 7207 raft_consensus.cc:1081] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 LEADER]: Signalling peer d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:37.392554 6100 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:58938
I20251209 03:04:37.392657 6100 raft_consensus.cc:493] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:37.392688 6100 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:37.393383 6100 raft_consensus.cc:515] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:37.393563 6100 leader_election.cc:290] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 6 election: Requested vote from peers b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597), 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151)
I20251209 03:04:37.393738 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 6 candidate_status { last_received { term: 5 index: 36 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:37.393743 6647 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b74f825019240c7950dcc767dd2e6a5" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 6 candidate_status { last_received { term: 5 index: 36 } } ignore_live_leader: true dest_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
I20251209 03:04:37.393819 6252 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:37.393832 6647 raft_consensus.cc:3055] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 LEADER]: Stepping down as leader of term 5
I20251209 03:04:37.393858 6647 raft_consensus.cc:740] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 03aecf9a33ec4b4ba130772d3c35d33e, State: Running, Role: LEADER
I20251209 03:04:37.393910 6647 consensus_queue.cc:260] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:37.394055 6647 raft_consensus.cc:3060] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 5 FOLLOWER]: Advancing to term 6
I20251209 03:04:37.394555 6252 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 6.
I20251209 03:04:37.394634 6647 raft_consensus.cc:2468] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 6.
I20251209 03:04:37.394764 6069 leader_election.cc:304] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:37.394869 7179 raft_consensus.cc:2804] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Leader election won for term 6
I20251209 03:04:37.394960 7179 raft_consensus.cc:697] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 6 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:37.395018 7179 consensus_queue.cc:237] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 5.36, Last appended by leader: 36, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:37.395563 6001 catalog_manager.cc:5654] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 5 to 6, leader changed from 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198) to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 6 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:37.829257 6252 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251209 03:04:37.829607 7179 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251209 03:04:37.830826 6647 raft_consensus.cc:1275] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 5 index: 36. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20251209 03:04:37.831125 7219 consensus_queue.cc:1048] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20251209 03:04:39.270804 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:39.271354 6100 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
new_leader_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:58930
I20251209 03:04:39.271442 6100 raft_consensus.cc:606] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Received request to transfer leadership to TS b2fc3e20641e45a4b62732385fd4e036
I20251209 03:04:39.271622 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:39.271700 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:39.658700 7223 raft_consensus.cc:993] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: : Instructing follower b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:39.658816 7223 raft_consensus.cc:1081] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Signalling peer b2fc3e20641e45a4b62732385fd4e036 to start an election
I20251209 03:04:39.659114 6252 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
from {username='slave'} at 127.0.0.1:35074
I20251209 03:04:39.659190 6252 raft_consensus.cc:493] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:39.659219 6252 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:39.659858 6252 raft_consensus.cc:515] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:39.660060 6252 leader_election.cc:290] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 4 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), 03aecf9a33ec4b4ba130772d3c35d33e (127.2.170.198:34151)
I20251209 03:04:39.660193 6100 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 4 candidate_status { last_received { term: 3 index: 34 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:39.660262 6100 raft_consensus.cc:3055] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Stepping down as leader of term 3
I20251209 03:04:39.660293 6100 raft_consensus.cc:740] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:39.660321 6646 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86b92235e7ad4a959951bdb816e39905" candidate_uuid: "b2fc3e20641e45a4b62732385fd4e036" candidate_term: 4 candidate_status { last_received { term: 3 index: 34 } } ignore_live_leader: true dest_uuid: "03aecf9a33ec4b4ba130772d3c35d33e"
I20251209 03:04:39.660358 6100 consensus_queue.cc:260] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 3.34, Last appended by leader: 34, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:39.660363 6646 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:39.660456 6100 raft_consensus.cc:3060] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 3 FOLLOWER]: Advancing to term 4
I20251209 03:04:39.661021 6100 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 4.
I20251209 03:04:39.661026 6646 raft_consensus.cc:2468] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2fc3e20641e45a4b62732385fd4e036 in term 4.
I20251209 03:04:39.661211 6220 leader_election.cc:304] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2fc3e20641e45a4b62732385fd4e036, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:39.661330 7162 raft_consensus.cc:2804] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Leader election won for term 4
I20251209 03:04:39.661430 7162 raft_consensus.cc:697] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Becoming Leader. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:39.661487 7162 consensus_queue.cc:237] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 3.34, Last appended by leader: 34, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } }
I20251209 03:04:39.662041 6001 catalog_manager.cc:5654] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 reported cstate change: term changed from 3 to 4, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195). New cstate: current_term: 4 leader_uuid: "b2fc3e20641e45a4b62732385fd4e036" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:40.044765 6100 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251209 03:04:40.045109 7162 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:40.046522 6646 raft_consensus.cc:1275] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Refusing update from remote peer b2fc3e20641e45a4b62732385fd4e036: Log matching property violated. Preceding OpId in replica: term: 3 index: 34. Preceding OpId from leader: term: 4 index: 35. (index mismatch)
I20251209 03:04:40.046923 7235 consensus_queue.cc:1048] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03aecf9a33ec4b4ba130772d3c35d33e" member_type: VOTER last_known_addr { host: "127.2.170.198" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20251209 03:04:41.271926 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:41.272511 6252 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
mode: GRACEFUL
new_leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:35098
I20251209 03:04:41.272603 6252 raft_consensus.cc:606] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 7 LEADER]: Received request to transfer leadership to TS a6fcaebe8f1f4a319f00f9666d8ba592
I20251209 03:04:41.272812 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:41.272881 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:41.619906 7235 raft_consensus.cc:993] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: : Instructing follower a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:41.619992 7235 raft_consensus.cc:1081] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 7 LEADER]: Signalling peer a6fcaebe8f1f4a319f00f9666d8ba592 to start an election
I20251209 03:04:41.620306 6175 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
from {username='slave'} at 127.0.0.1:43158
I20251209 03:04:41.620399 6175 raft_consensus.cc:493] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:41.620421 6175 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 7 FOLLOWER]: Advancing to term 8
I20251209 03:04:41.621081 6175 raft_consensus.cc:515] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:41.621261 6175 leader_election.cc:290] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 8 election: Requested vote from peers b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195:34597), 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:41.621408 6496 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 8 candidate_status { last_received { term: 7 index: 38 } } ignore_live_leader: true dest_uuid: "37b7f916c7c843dfa479e106f762cfff"
I20251209 03:04:41.621408 6252 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5ef8ec5789d64b8f931598cfcded95f2" candidate_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" candidate_term: 8 candidate_status { last_received { term: 7 index: 38 } } ignore_live_leader: true dest_uuid: "b2fc3e20641e45a4b62732385fd4e036"
I20251209 03:04:41.621511 6496 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 7 FOLLOWER]: Advancing to term 8
I20251209 03:04:41.621520 6252 raft_consensus.cc:3055] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 7 LEADER]: Stepping down as leader of term 7
I20251209 03:04:41.621559 6252 raft_consensus.cc:740] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b2fc3e20641e45a4b62732385fd4e036, State: Running, Role: LEADER
I20251209 03:04:41.621615 6252 consensus_queue.cc:260] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 7.38, Last appended by leader: 38, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:41.621727 6252 raft_consensus.cc:3060] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 7 FOLLOWER]: Advancing to term 8
I20251209 03:04:41.622327 6252 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 8.
I20251209 03:04:41.622327 6496 raft_consensus.cc:2468] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fcaebe8f1f4a319f00f9666d8ba592 in term 8.
I20251209 03:04:41.622532 6143 leader_election.cc:304] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b7f916c7c843dfa479e106f762cfff, a6fcaebe8f1f4a319f00f9666d8ba592; no voters:
I20251209 03:04:41.622645 7144 raft_consensus.cc:2804] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 8 FOLLOWER]: Leader election won for term 8
I20251209 03:04:41.622761 7144 raft_consensus.cc:697] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 8 LEADER]: Becoming Leader. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:41.622812 7144 consensus_queue.cc:237] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 7.38, Last appended by leader: 38, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:41.623323 6001 catalog_manager.cc:5654] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 reported cstate change: term changed from 7 to 8, leader changed from b2fc3e20641e45a4b62732385fd4e036 (127.2.170.195) to a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194). New cstate: current_term: 8 leader_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" committed_config { opid_index: 36 OBSOLETE_local: false peers { permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:42.047309 6496 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 8 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 7 index: 38. Preceding OpId from leader: term: 8 index: 39. (index mismatch)
I20251209 03:04:42.047612 7144 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20251209 03:04:42.049093 6252 raft_consensus.cc:1275] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 8 FOLLOWER]: Refusing update from remote peer a6fcaebe8f1f4a319f00f9666d8ba592: Log matching property violated. Preceding OpId in replica: term: 7 index: 38. Preceding OpId from leader: term: 8 index: 39. (index mismatch)
I20251209 03:04:42.049358 7144 consensus_queue.cc:1048] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2fc3e20641e45a4b62732385fd4e036" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20251209 03:04:43.273032 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:43.273461 6175 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "f9a52de6742a4e508e06374063561398"
dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
mode: GRACEFUL
new_leader_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:43196
I20251209 03:04:43.273551 6175 raft_consensus.cc:606] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 LEADER]: Received request to transfer leadership to TS d36845214167428eb93883ecbe8f5385
I20251209 03:04:43.273751 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:43.273814 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:43.292485 7251 raft_consensus.cc:993] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: : Instructing follower d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:43.292572 7251 raft_consensus.cc:1081] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 LEADER]: Signalling peer d36845214167428eb93883ecbe8f5385 to start an election
I20251209 03:04:43.292758 6100 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "f9a52de6742a4e508e06374063561398"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
from {username='slave'} at 127.0.0.1:58880
I20251209 03:04:43.292855 6100 raft_consensus.cc:493] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:43.292882 6100 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:43.293737 6100 raft_consensus.cc:515] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:43.293926 6100 leader_election.cc:290] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 5 election: Requested vote from peers a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307), 37b7f916c7c843dfa479e106f762cfff (127.2.170.196:43979)
I20251209 03:04:43.294066 6496 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "37b7f916c7c843dfa479e106f762cfff"
I20251209 03:04:43.294178 6496 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:43.294937 6496 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 5.
I20251209 03:04:43.294063 6175 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f9a52de6742a4e508e06374063561398" candidate_uuid: "d36845214167428eb93883ecbe8f5385" candidate_term: 5 candidate_status { last_received { term: 4 index: 33 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:43.295102 6067 leader_election.cc:304] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b7f916c7c843dfa479e106f762cfff, d36845214167428eb93883ecbe8f5385; no voters:
I20251209 03:04:43.295140 6175 raft_consensus.cc:3055] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 LEADER]: Stepping down as leader of term 4
I20251209 03:04:43.295182 6175 raft_consensus.cc:740] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a6fcaebe8f1f4a319f00f9666d8ba592, State: Running, Role: LEADER
I20251209 03:04:43.295205 7223 raft_consensus.cc:2804] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Leader election won for term 5
I20251209 03:04:43.295243 6175 consensus_queue.cc:260] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:43.295302 7223 raft_consensus.cc:697] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 5 LEADER]: Becoming Leader. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:43.295328 6175 raft_consensus.cc:3060] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 4 FOLLOWER]: Advancing to term 5
I20251209 03:04:43.295343 7223 consensus_queue.cc:237] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 4.33, Last appended by leader: 33, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } }
I20251209 03:04:43.295897 6175 raft_consensus.cc:2468] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36845214167428eb93883ecbe8f5385 in term 5.
I20251209 03:04:43.296285 5999 catalog_manager.cc:5654] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 reported cstate change: term changed from 4 to 5, leader changed from a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194) to d36845214167428eb93883ecbe8f5385 (127.2.170.193). New cstate: current_term: 5 leader_uuid: "d36845214167428eb93883ecbe8f5385" committed_config { opid_index: 32 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:43.701195 6175 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251209 03:04:43.701478 7223 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:43.702654 6496 raft_consensus.cc:1275] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 5 FOLLOWER]: Refusing update from remote peer d36845214167428eb93883ecbe8f5385: Log matching property violated. Preceding OpId in replica: term: 4 index: 33. Preceding OpId from leader: term: 5 index: 34. (index mismatch)
I20251209 03:04:43.702821 7253 consensus_queue.cc:1048] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b7f916c7c843dfa479e106f762cfff" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 43979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20251209 03:04:45.273977 6055 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:45.274439 6100 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "7ead69a2942346dca25124b075136d12"
dest_uuid: "d36845214167428eb93883ecbe8f5385"
mode: GRACEFUL
new_leader_uuid: "2551ae093c914660b7f6e42bbea920fa"
from {username='slave'} at 127.0.0.1:58930
I20251209 03:04:45.274528 6100 raft_consensus.cc:606] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Received request to transfer leadership to TS 2551ae093c914660b7f6e42bbea920fa
I20251209 03:04:45.274677 6055 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:45.274757 6055 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:45.500845 7265 raft_consensus.cc:993] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: : Instructing follower 2551ae093c914660b7f6e42bbea920fa to start an election
I20251209 03:04:45.500964 7265 raft_consensus.cc:1081] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Signalling peer 2551ae093c914660b7f6e42bbea920fa to start an election
I20251209 03:04:45.501150 6571 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "7ead69a2942346dca25124b075136d12"
dest_uuid: "2551ae093c914660b7f6e42bbea920fa"
from {username='slave'} at 127.0.0.1:45344
I20251209 03:04:45.501238 6571 raft_consensus.cc:493] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:45.501259 6571 raft_consensus.cc:3060] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:45.501904 6571 raft_consensus.cc:515] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:45.502061 6571 leader_election.cc:290] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [CANDIDATE]: Term 2 election: Requested vote from peers d36845214167428eb93883ecbe8f5385 (127.2.170.193:43899), a6fcaebe8f1f4a319f00f9666d8ba592 (127.2.170.194:42307)
I20251209 03:04:45.502219 6100 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ead69a2942346dca25124b075136d12" candidate_uuid: "2551ae093c914660b7f6e42bbea920fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "d36845214167428eb93883ecbe8f5385"
I20251209 03:04:45.502219 6180 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7ead69a2942346dca25124b075136d12" candidate_uuid: "2551ae093c914660b7f6e42bbea920fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 31 } } ignore_live_leader: true dest_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592"
I20251209 03:04:45.502321 6100 raft_consensus.cc:3055] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:45.502324 6180 raft_consensus.cc:3060] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:45.502353 6100 raft_consensus.cc:740] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d36845214167428eb93883ecbe8f5385, State: Running, Role: LEADER
I20251209 03:04:45.502436 6100 consensus_queue.cc:260] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:45.502548 6100 raft_consensus.cc:3060] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:45.503067 6180 raft_consensus.cc:2468] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2551ae093c914660b7f6e42bbea920fa in term 2.
I20251209 03:04:45.503156 6100 raft_consensus.cc:2468] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2551ae093c914660b7f6e42bbea920fa in term 2.
I20251209 03:04:45.503254 6537 leader_election.cc:304] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2551ae093c914660b7f6e42bbea920fa, a6fcaebe8f1f4a319f00f9666d8ba592; no voters:
I20251209 03:04:45.503351 7224 raft_consensus.cc:2804] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:45.503463 7224 raft_consensus.cc:697] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEADER]: Becoming Leader. State: Replica: 2551ae093c914660b7f6e42bbea920fa, State: Running, Role: LEADER
I20251209 03:04:45.503530 7224 consensus_queue.cc:237] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 1.31, Last appended by leader: 31, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } }
I20251209 03:04:45.503971 5999 catalog_manager.cc:5654] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa reported cstate change: term changed from 1 to 2, leader changed from d36845214167428eb93883ecbe8f5385 (127.2.170.193) to 2551ae093c914660b7f6e42bbea920fa (127.2.170.197). New cstate: current_term: 2 leader_uuid: "2551ae093c914660b7f6e42bbea920fa" committed_config { opid_index: 31 OBSOLETE_local: false peers { permanent_uuid: "d36845214167428eb93883ecbe8f5385" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fcaebe8f1f4a319f00f9666d8ba592" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 42307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2551ae093c914660b7f6e42bbea920fa" member_type: VOTER last_known_addr { host: "127.2.170.197" port: 41607 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/auto_rebalancer-test.cc:196: Failure
Expected equality of these values:
0
NumMovesScheduled(leader_idx, BalanceThreadType::LEADER_REBALANCE)
Which is: 1
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/auto_rebalancer-test.cc:575: Failure
Expected: CheckNoLeaderMovesScheduled() doesn't generate new fatal failures in the current thread.
Actual: it does.
I20251209 03:04:45.565295 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:04:45.568646 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:45.568778 2731 tablet_replica.cc:333] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:45.568842 2731 raft_consensus.cc:2243] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 6 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.568964 2731 raft_consensus.cc:2272] T 2b74f825019240c7950dcc767dd2e6a5 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.569128 2731 tablet_replica.cc:333] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:45.569161 2731 raft_consensus.cc:2243] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.569181 2731 raft_consensus.cc:2272] T 86b92235e7ad4a959951bdb816e39905 P d36845214167428eb93883ecbe8f5385 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.569259 2731 tablet_replica.cc:333] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:45.569283 2731 raft_consensus.cc:2243] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.569314 2731 raft_consensus.cc:2272] T c5b028a080314d919052909cd10db307 P d36845214167428eb93883ecbe8f5385 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.569388 2731 tablet_replica.cc:333] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:45.569411 2731 raft_consensus.cc:2243] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.569432 2731 raft_consensus.cc:2272] T 7ead69a2942346dca25124b075136d12 P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.569506 2731 tablet_replica.cc:333] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:45.569530 2731 raft_consensus.cc:2243] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 5 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.569566 2731 raft_consensus.cc:2272] T f9a52de6742a4e508e06374063561398 P d36845214167428eb93883ecbe8f5385 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.569640 2731 tablet_replica.cc:333] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385: stopping tablet replica
I20251209 03:04:45.569662 2731 raft_consensus.cc:2243] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.569772 2731 raft_consensus.cc:2272] T 0e6b3f62d7894c69b4cf99bf1ad942ce P d36845214167428eb93883ecbe8f5385 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.582353 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:04:45.584004 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:04:45.587602 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:45.587734 2731 tablet_replica.cc:333] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:45.587801 2731 raft_consensus.cc:2243] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.587894 2731 raft_consensus.cc:2272] T a9a5e7d1a2de4913a09dfeb084d79ae1 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.588066 2731 tablet_replica.cc:333] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:45.588115 2731 raft_consensus.cc:2243] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.588201 2731 raft_consensus.cc:2272] T f71e8ee2ce1242608207622b267d96ae P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.588361 2731 tablet_replica.cc:333] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:45.588395 2731 raft_consensus.cc:2243] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.588469 2731 raft_consensus.cc:2272] T 7ead69a2942346dca25124b075136d12 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.588582 2731 tablet_replica.cc:333] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:45.588624 2731 raft_consensus.cc:2243] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 8 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.588680 2731 raft_consensus.cc:2272] T 5ef8ec5789d64b8f931598cfcded95f2 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 8 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.588788 2731 tablet_replica.cc:333] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:45.588825 2731 raft_consensus.cc:2243] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.588871 2731 raft_consensus.cc:2272] T 0e6b3f62d7894c69b4cf99bf1ad942ce P a6fcaebe8f1f4a319f00f9666d8ba592 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.588989 2731 tablet_replica.cc:333] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592: stopping tablet replica
I20251209 03:04:45.589018 2731 raft_consensus.cc:2243] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.589044 2731 raft_consensus.cc:2272] T f9a52de6742a4e508e06374063561398 P a6fcaebe8f1f4a319f00f9666d8ba592 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.591198 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:04:45.592615 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:04:45.595993 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:45.596094 2731 tablet_replica.cc:333] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:45.596169 2731 raft_consensus.cc:2243] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.596230 2731 raft_consensus.cc:2272] T 439ea3fec68444aeb8cc383b977a7f48 P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.596354 2731 tablet_replica.cc:333] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:45.596387 2731 raft_consensus.cc:2243] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 8 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.596410 2731 raft_consensus.cc:2272] T 5ef8ec5789d64b8f931598cfcded95f2 P b2fc3e20641e45a4b62732385fd4e036 [term 8 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.596535 2731 tablet_replica.cc:333] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:45.596566 2731 raft_consensus.cc:2243] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.596583 2731 raft_consensus.cc:2272] T 2b74f825019240c7950dcc767dd2e6a5 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.596657 2731 tablet_replica.cc:333] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:45.596680 2731 raft_consensus.cc:2243] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.596711 2731 raft_consensus.cc:2272] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P b2fc3e20641e45a4b62732385fd4e036 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.596797 2731 tablet_replica.cc:333] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:45.596815 2731 raft_consensus.cc:2243] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 4 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.596848 2731 raft_consensus.cc:2272] T 86b92235e7ad4a959951bdb816e39905 P b2fc3e20641e45a4b62732385fd4e036 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.596918 2731 tablet_replica.cc:333] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036: stopping tablet replica
I20251209 03:04:45.596946 2731 raft_consensus.cc:2243] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 6 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.596980 2731 raft_consensus.cc:2272] T c5b028a080314d919052909cd10db307 P b2fc3e20641e45a4b62732385fd4e036 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.609105 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:04:45.610647 2731 tablet_server.cc:178] TabletServer@127.2.170.196:0 shutting down...
I20251209 03:04:45.613327 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:45.613433 2731 tablet_replica.cc:333] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff: stopping tablet replica
I20251209 03:04:45.613494 2731 raft_consensus.cc:2243] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.613564 2731 raft_consensus.cc:2272] T cf139ecb17be4ec58d5c23e86eab88a4 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.613752 2731 tablet_replica.cc:333] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff: stopping tablet replica
I20251209 03:04:45.613794 2731 raft_consensus.cc:2243] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 8 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.613835 2731 raft_consensus.cc:2272] T 5ef8ec5789d64b8f931598cfcded95f2 P 37b7f916c7c843dfa479e106f762cfff [term 8 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.613926 2731 tablet_replica.cc:333] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff: stopping tablet replica
I20251209 03:04:45.613953 2731 raft_consensus.cc:2243] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.613986 2731 raft_consensus.cc:2272] T f71e8ee2ce1242608207622b267d96ae P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.614058 2731 tablet_replica.cc:333] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff: stopping tablet replica
I20251209 03:04:45.614082 2731 raft_consensus.cc:2243] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 5 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.614113 2731 raft_consensus.cc:2272] T f9a52de6742a4e508e06374063561398 P 37b7f916c7c843dfa479e106f762cfff [term 5 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.614194 2731 tablet_replica.cc:333] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff: stopping tablet replica
I20251209 03:04:45.614218 2731 raft_consensus.cc:2243] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.614259 2731 raft_consensus.cc:2272] T 439ea3fec68444aeb8cc383b977a7f48 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.614333 2731 tablet_replica.cc:333] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff: stopping tablet replica
I20251209 03:04:45.614355 2731 raft_consensus.cc:2243] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.614387 2731 raft_consensus.cc:2272] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 37b7f916c7c843dfa479e106f762cfff [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.627004 2731 tablet_server.cc:195] TabletServer@127.2.170.196:0 shutdown complete.
I20251209 03:04:45.628405 2731 tablet_server.cc:178] TabletServer@127.2.170.197:0 shutting down...
I20251209 03:04:45.633060 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:45.633165 2731 tablet_replica.cc:333] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa: stopping tablet replica
I20251209 03:04:45.633232 2731 raft_consensus.cc:2243] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.633287 2731 raft_consensus.cc:2272] T 439ea3fec68444aeb8cc383b977a7f48 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.633432 2731 tablet_replica.cc:333] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa: stopping tablet replica
I20251209 03:04:45.633466 2731 raft_consensus.cc:2243] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.633512 2731 raft_consensus.cc:2272] T cf139ecb17be4ec58d5c23e86eab88a4 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.633591 2731 tablet_replica.cc:333] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa: stopping tablet replica
I20251209 03:04:45.633616 2731 raft_consensus.cc:2243] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.633656 2731 raft_consensus.cc:2272] T 0e6b3f62d7894c69b4cf99bf1ad942ce P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.633795 2731 tablet_replica.cc:333] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa: stopping tablet replica
I20251209 03:04:45.633833 2731 raft_consensus.cc:2243] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.633873 2731 raft_consensus.cc:2272] T a9a5e7d1a2de4913a09dfeb084d79ae1 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.633960 2731 tablet_replica.cc:333] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa: stopping tablet replica
I20251209 03:04:45.633985 2731 raft_consensus.cc:2243] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.634028 2731 raft_consensus.cc:2272] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.634115 2731 tablet_replica.cc:333] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: stopping tablet replica
I20251209 03:04:45.634140 2731 raft_consensus.cc:2243] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.634176 2731 pending_rounds.cc:70] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: Trying to abort 1 pending ops.
I20251209 03:04:45.634192 2731 pending_rounds.cc:77] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa: Aborting op as it isn't in flight: id { term: 2 index: 32 } timestamp: 7230461892623007744 op_type: NO_OP noop_request { }
I20251209 03:04:45.634239 2731 raft_consensus.cc:2889] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251209 03:04:45.634259 2731 raft_consensus.cc:2272] T 7ead69a2942346dca25124b075136d12 P 2551ae093c914660b7f6e42bbea920fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.646255 2731 tablet_server.cc:195] TabletServer@127.2.170.197:0 shutdown complete.
I20251209 03:04:45.647513 2731 tablet_server.cc:178] TabletServer@127.2.170.198:0 shutting down...
I20251209 03:04:45.650298 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:45.650435 2731 tablet_replica.cc:333] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e: stopping tablet replica
I20251209 03:04:45.650507 2731 raft_consensus.cc:2243] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.650552 2731 raft_consensus.cc:2272] T 2beec1d9e1ff42cea8ff5a8e9024a6fe P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.650693 2731 tablet_replica.cc:333] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e: stopping tablet replica
I20251209 03:04:45.650722 2731 raft_consensus.cc:2243] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.650745 2731 raft_consensus.cc:2272] T 2b74f825019240c7950dcc767dd2e6a5 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.650825 2731 tablet_replica.cc:333] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e: stopping tablet replica
I20251209 03:04:45.650851 2731 raft_consensus.cc:2243] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.650900 2731 raft_consensus.cc:2272] T 86b92235e7ad4a959951bdb816e39905 P 03aecf9a33ec4b4ba130772d3c35d33e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.650983 2731 tablet_replica.cc:333] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e: stopping tablet replica
I20251209 03:04:45.651008 2731 raft_consensus.cc:2243] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:45.651041 2731 raft_consensus.cc:2272] T c5b028a080314d919052909cd10db307 P 03aecf9a33ec4b4ba130772d3c35d33e [term 6 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.651109 2731 tablet_replica.cc:333] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e: stopping tablet replica
I20251209 03:04:45.651136 2731 raft_consensus.cc:2243] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.651178 2731 raft_consensus.cc:2272] T cf139ecb17be4ec58d5c23e86eab88a4 P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.651254 2731 tablet_replica.cc:333] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e: stopping tablet replica
I20251209 03:04:45.651279 2731 raft_consensus.cc:2243] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.651316 2731 raft_consensus.cc:2272] T f71e8ee2ce1242608207622b267d96ae P 03aecf9a33ec4b4ba130772d3c35d33e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.663080 2731 tablet_server.cc:195] TabletServer@127.2.170.198:0 shutdown complete.
I20251209 03:04:45.664532 2731 master.cc:561] Master@127.2.170.254:39481 shutting down...
I20251209 03:04:45.667333 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:04:45.667403 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:45.667423 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6f5640f467eb4d63bcca994b2fa93dd8: stopping tablet replica
I20251209 03:04:45.679280 2731 master.cc:583] Master@127.2.170.254:39481 shutdown complete.
I20251209 03:04:45.681238 2731 test_util.cc:183] -----------------------------------------------
I20251209 03:04:45.681301 2731 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestMaxMovesPerServer.1765249356537226-2731-0
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer (36488 ms)
[ RUN ] AutoRebalancerTest.AutoRebalancingUnstableCluster
I20251209 03:04:45.681788 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:46019
I20251209 03:04:45.681934 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:45.683287 7275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:45.683398 7272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:45.683439 7273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:45.683495 2731 server_base.cc:1047] running on GCE node
I20251209 03:04:45.683588 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:45.683614 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:45.683625 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249485683626 us; error 0 us; skew 500 ppm
I20251209 03:04:45.684152 2731 webserver.cc:492] Webserver started at http://127.2.170.254:43643/ using document root <none> and password file <none>
I20251209 03:04:45.684221 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:45.684252 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:45.684283 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:45.684478 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "7cb7da8637bd49748adf681e73ff1ae4"
format_stamp: "Formatted at 2025-12-09 03:04:45 on dist-test-slave-4cb0"
I20251209 03:04:45.685235 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:45.685611 7280 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.685794 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.685834 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "7cb7da8637bd49748adf681e73ff1ae4"
format_stamp: "Formatted at 2025-12-09 03:04:45 on dist-test-slave-4cb0"
I20251209 03:04:45.685901 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:45.709090 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:45.709282 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:45.711668 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:46019
I20251209 03:04:45.717761 7332 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:46019 every 8 connection(s)
I20251209 03:04:45.717913 7333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:45.718873 7333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4: Bootstrap starting.
I20251209 03:04:45.719141 7333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:45.719605 7333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4: No bootstrap required, opened a new log
I20251209 03:04:45.719741 7333 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7da8637bd49748adf681e73ff1ae4" member_type: VOTER }
I20251209 03:04:45.719794 7333 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:45.719811 7333 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb7da8637bd49748adf681e73ff1ae4, State: Initialized, Role: FOLLOWER
I20251209 03:04:45.719857 7333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7da8637bd49748adf681e73ff1ae4" member_type: VOTER }
I20251209 03:04:45.719915 7333 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:04:45.719933 7333 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:04:45.719952 7333 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:45.720420 7333 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7da8637bd49748adf681e73ff1ae4" member_type: VOTER }
I20251209 03:04:45.720474 7333 leader_election.cc:304] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cb7da8637bd49748adf681e73ff1ae4; no voters:
I20251209 03:04:45.720572 7333 leader_election.cc:290] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:04:45.720628 7336 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:45.720717 7333 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:04:45.720733 7336 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 1 LEADER]: Becoming Leader. State: Replica: 7cb7da8637bd49748adf681e73ff1ae4, State: Running, Role: LEADER
I20251209 03:04:45.720772 7336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7da8637bd49748adf681e73ff1ae4" member_type: VOTER }
I20251209 03:04:45.720983 7338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb7da8637bd49748adf681e73ff1ae4. Latest consensus state: current_term: 1 leader_uuid: "7cb7da8637bd49748adf681e73ff1ae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7da8637bd49748adf681e73ff1ae4" member_type: VOTER } }
I20251209 03:04:45.720971 7337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cb7da8637bd49748adf681e73ff1ae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7da8637bd49748adf681e73ff1ae4" member_type: VOTER } }
I20251209 03:04:45.721064 7338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [sys.catalog]: This master's current role is: LEADER
I20251209 03:04:45.721072 7337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [sys.catalog]: This master's current role is: LEADER
I20251209 03:04:45.721261 7340 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:04:45.721390 7340 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:04:45.721939 7340 catalog_manager.cc:1357] Generated new cluster ID: f0e5da7d3e754f06b160edfd490f996f
I20251209 03:04:45.721987 7340 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:04:45.722005 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:04:45.726464 7340 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:04:45.726869 7340 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:04:45.731712 7340 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4: Generated new TSK 0
I20251209 03:04:45.731786 7340 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:04:45.737665 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:45.738898 7354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:45.738960 7357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:45.739041 7355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:45.739248 2731 server_base.cc:1047] running on GCE node
I20251209 03:04:45.739326 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:45.739356 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:45.739378 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249485739379 us; error 0 us; skew 500 ppm
I20251209 03:04:45.740016 2731 webserver.cc:492] Webserver started at http://127.2.170.193:33449/ using document root <none> and password file <none>
I20251209 03:04:45.740090 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:45.740134 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:45.740180 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:45.740422 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "d7a093a649e84cbb90ce0739ec97951a"
format_stamp: "Formatted at 2025-12-09 03:04:45 on dist-test-slave-4cb0"
I20251209 03:04:45.741179 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:45.741549 7362 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.741696 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.741741 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "d7a093a649e84cbb90ce0739ec97951a"
format_stamp: "Formatted at 2025-12-09 03:04:45 on dist-test-slave-4cb0"
I20251209 03:04:45.741784 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:45.748586 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:45.748757 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:45.748901 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:45.749081 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:45.749107 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.749135 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:45.749158 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.751734 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:32885
I20251209 03:04:45.751785 7425 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:32885 every 8 connection(s)
I20251209 03:04:45.752074 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:45.753224 7430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:45.753301 7431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:45.753950 7433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:45.753995 2731 server_base.cc:1047] running on GCE node
I20251209 03:04:45.754148 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:45.754181 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:45.754194 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249485754194 us; error 0 us; skew 500 ppm
I20251209 03:04:45.754787 2731 webserver.cc:492] Webserver started at http://127.2.170.194:33529/ using document root <none> and password file <none>
I20251209 03:04:45.754871 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:45.754914 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:45.754958 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:45.755225 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e"
format_stamp: "Formatted at 2025-12-09 03:04:45 on dist-test-slave-4cb0"
I20251209 03:04:45.755748 7426 heartbeater.cc:344] Connected to a master server at 127.2.170.254:46019
I20251209 03:04:45.755816 7426 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:45.755925 7426 heartbeater.cc:507] Master 127.2.170.254:46019 requested a full tablet report, sending...
I20251209 03:04:45.756140 7297 ts_manager.cc:194] Registered new tserver with Master: d7a093a649e84cbb90ce0739ec97951a (127.2.170.193:32885)
I20251209 03:04:45.756162 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:45.756594 7438 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.756726 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.756755 7297 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38548
I20251209 03:04:45.756770 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e"
format_stamp: "Formatted at 2025-12-09 03:04:45 on dist-test-slave-4cb0"
I20251209 03:04:45.756841 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:45.765611 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:45.765858 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:45.765995 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:45.766165 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:45.766191 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.766261 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:45.766289 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.768846 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:44245
I20251209 03:04:45.768900 7501 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:44245 every 8 connection(s)
I20251209 03:04:45.769238 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:45.770535 7505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:45.770582 7506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:45.770682 2731 server_base.cc:1047] running on GCE node
W20251209 03:04:45.770853 7508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:45.770965 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:45.771005 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:45.771023 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249485771022 us; error 0 us; skew 500 ppm
I20251209 03:04:45.771661 2731 webserver.cc:492] Webserver started at http://127.2.170.195:38447/ using document root <none> and password file <none>
I20251209 03:04:45.771736 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:45.771780 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:45.771822 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:45.772095 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "89c1072203c242959f590180e345a328"
format_stamp: "Formatted at 2025-12-09 03:04:45 on dist-test-slave-4cb0"
I20251209 03:04:45.772114 7502 heartbeater.cc:344] Connected to a master server at 127.2.170.254:46019
I20251209 03:04:45.772187 7502 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:45.772281 7502 heartbeater.cc:507] Master 127.2.170.254:46019 requested a full tablet report, sending...
I20251209 03:04:45.772496 7297 ts_manager.cc:194] Registered new tserver with Master: 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245)
I20251209 03:04:45.772951 7297 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38558
I20251209 03:04:45.773106 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:45.773627 7513 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.773777 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251209 03:04:45.773824 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "89c1072203c242959f590180e345a328"
format_stamp: "Formatted at 2025-12-09 03:04:45 on dist-test-slave-4cb0"
I20251209 03:04:45.773869 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:45.792738 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:45.792932 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:45.793074 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:45.793280 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:45.793308 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.793336 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:45.793360 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:45.795961 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:39919
I20251209 03:04:45.796023 7576 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:39919 every 8 connection(s)
I20251209 03:04:45.798983 7577 heartbeater.cc:344] Connected to a master server at 127.2.170.254:46019
I20251209 03:04:45.799057 7577 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:45.799161 7577 heartbeater.cc:507] Master 127.2.170.254:46019 requested a full tablet report, sending...
I20251209 03:04:45.799423 7297 ts_manager.cc:194] Registered new tserver with Master: 89c1072203c242959f590180e345a328 (127.2.170.195:39919)
I20251209 03:04:45.799516 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003388017s
I20251209 03:04:45.799962 7297 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38560
I20251209 03:04:46.757321 7426 heartbeater.cc:499] Master 127.2.170.254:46019 was elected leader, sending a full tablet report...
I20251209 03:04:46.773458 7502 heartbeater.cc:499] Master 127.2.170.254:46019 was elected leader, sending a full tablet report...
I20251209 03:04:46.800586 7577 heartbeater.cc:499] Master 127.2.170.254:46019 was elected leader, sending a full tablet report...
I20251209 03:04:46.824554 2731 test_util.cc:276] Using random seed: 748299366
I20251209 03:04:46.829838 7297 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38574:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\252\252\252*\004\001\000TUUU""\004\001\000\252\252\252*\004\001\000TUUU"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:04:46.830022 7297 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:04:46.835355 7466 tablet_service.cc:1505] Processing CreateTablet for tablet 75ce916182974eb29ee2dac0d1c64068 (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251209 03:04:46.835355 7541 tablet_service.cc:1505] Processing CreateTablet for tablet 75ce916182974eb29ee2dac0d1c64068 (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251209 03:04:46.835387 7539 tablet_service.cc:1505] Processing CreateTablet for tablet 57e328b687194376b21c2b3790a1f309 (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251209 03:04:46.835521 7466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ce916182974eb29ee2dac0d1c64068. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.835552 7539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57e328b687194376b21c2b3790a1f309. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.835722 7389 tablet_service.cc:1505] Processing CreateTablet for tablet 6f10c10456ac4cb2a1abaa81da3f11cd (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251209 03:04:46.835783 7388 tablet_service.cc:1505] Processing CreateTablet for tablet 57e328b687194376b21c2b3790a1f309 (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251209 03:04:46.835738 7390 tablet_service.cc:1505] Processing CreateTablet for tablet 75ce916182974eb29ee2dac0d1c64068 (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION VALUES < 715827882
I20251209 03:04:46.835858 7389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f10c10456ac4cb2a1abaa81da3f11cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.835939 7390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ce916182974eb29ee2dac0d1c64068. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.835355 7540 tablet_service.cc:1505] Processing CreateTablet for tablet 6f10c10456ac4cb2a1abaa81da3f11cd (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251209 03:04:46.836402 7540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f10c10456ac4cb2a1abaa81da3f11cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.836454 7464 tablet_service.cc:1505] Processing CreateTablet for tablet 57e328b687194376b21c2b3790a1f309 (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION 1431655764 <= VALUES
I20251209 03:04:46.836416 7465 tablet_service.cc:1505] Processing CreateTablet for tablet 6f10c10456ac4cb2a1abaa81da3f11cd (DEFAULT_TABLE table=test-workload [id=f6c0eeda38cd415996e49d3429315fea]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1431655764
I20251209 03:04:46.836532 7464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57e328b687194376b21c2b3790a1f309. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.837316 7465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f10c10456ac4cb2a1abaa81da3f11cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.837342 7598 tablet_bootstrap.cc:492] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a: Bootstrap starting.
I20251209 03:04:46.837504 7388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57e328b687194376b21c2b3790a1f309. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.837702 7598 tablet_bootstrap.cc:654] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.838335 7600 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Bootstrap starting.
I20251209 03:04:46.838724 7597 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328: Bootstrap starting.
I20251209 03:04:46.838747 7600 tablet_bootstrap.cc:654] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.839166 7597 tablet_bootstrap.cc:654] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.839218 7541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ce916182974eb29ee2dac0d1c64068. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:46.839704 7600 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: No bootstrap required, opened a new log
I20251209 03:04:46.839774 7600 ts_tablet_manager.cc:1403] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:46.839778 7597 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328: No bootstrap required, opened a new log
I20251209 03:04:46.839880 7597 ts_tablet_manager.cc:1403] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:46.839972 7600 raft_consensus.cc:359] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.840032 7600 raft_consensus.cc:385] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.840018 7597 raft_consensus.cc:359] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.840050 7600 raft_consensus.cc:740] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7928ecfc0bab4a6e88b37cdb44b48d2e, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.840070 7597 raft_consensus.cc:385] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.840087 7597 raft_consensus.cc:740] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89c1072203c242959f590180e345a328, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.840118 7600 consensus_queue.cc:260] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.840130 7597 consensus_queue.cc:260] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.840272 7597 ts_tablet_manager.cc:1434] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.840260 7600 ts_tablet_manager.cc:1434] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.840373 7600 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e: Bootstrap starting.
I20251209 03:04:46.840716 7600 tablet_bootstrap.cc:654] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.840965 7597 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: Bootstrap starting.
I20251209 03:04:46.841320 7597 tablet_bootstrap.cc:654] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.841915 7597 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: No bootstrap required, opened a new log
I20251209 03:04:46.841965 7597 ts_tablet_manager.cc:1403] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:46.842100 7597 raft_consensus.cc:359] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.842172 7597 raft_consensus.cc:385] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.842177 7598 tablet_bootstrap.cc:492] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a: No bootstrap required, opened a new log
I20251209 03:04:46.842203 7597 raft_consensus.cc:740] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89c1072203c242959f590180e345a328, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.842254 7598 ts_tablet_manager.cc:1403] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20251209 03:04:46.842285 7600 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e: No bootstrap required, opened a new log
I20251209 03:04:46.842336 7600 ts_tablet_manager.cc:1403] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:04:46.842319 7597 consensus_queue.cc:260] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.842406 7597 ts_tablet_manager.cc:1434] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.842432 7598 raft_consensus.cc:359] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.842454 7597 tablet_bootstrap.cc:492] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328: Bootstrap starting.
I20251209 03:04:46.842482 7598 raft_consensus.cc:385] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.842530 7598 raft_consensus.cc:740] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a093a649e84cbb90ce0739ec97951a, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.842617 7598 consensus_queue.cc:260] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.842798 7598 ts_tablet_manager.cc:1434] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.842836 7597 tablet_bootstrap.cc:654] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.842912 7598 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a: Bootstrap starting.
I20251209 03:04:46.843272 7598 tablet_bootstrap.cc:654] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.842450 7600 raft_consensus.cc:359] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.843370 7600 raft_consensus.cc:385] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.843395 7597 tablet_bootstrap.cc:492] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328: No bootstrap required, opened a new log
I20251209 03:04:46.843434 7600 raft_consensus.cc:740] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7928ecfc0bab4a6e88b37cdb44b48d2e, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.843464 7597 ts_tablet_manager.cc:1403] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:46.843499 7600 consensus_queue.cc:260] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.843597 7597 raft_consensus.cc:359] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.843641 7600 ts_tablet_manager.cc:1434] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:04:46.843650 7597 raft_consensus.cc:385] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.843708 7600 tablet_bootstrap.cc:492] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Bootstrap starting.
I20251209 03:04:46.843720 7597 raft_consensus.cc:740] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89c1072203c242959f590180e345a328, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.843756 7597 consensus_queue.cc:260] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.843837 7597 ts_tablet_manager.cc:1434] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.843988 7600 tablet_bootstrap.cc:654] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.844472 7600 tablet_bootstrap.cc:492] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e: No bootstrap required, opened a new log
I20251209 03:04:46.844522 7600 ts_tablet_manager.cc:1403] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:46.844584 7598 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a: No bootstrap required, opened a new log
I20251209 03:04:46.844616 7598 ts_tablet_manager.cc:1403] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:04:46.844650 7600 raft_consensus.cc:359] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.844694 7600 raft_consensus.cc:385] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.844707 7600 raft_consensus.cc:740] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7928ecfc0bab4a6e88b37cdb44b48d2e, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.844712 7598 raft_consensus.cc:359] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.844736 7600 consensus_queue.cc:260] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.844750 7598 raft_consensus.cc:385] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.844769 7598 raft_consensus.cc:740] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a093a649e84cbb90ce0739ec97951a, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.844789 7600 ts_tablet_manager.cc:1434] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.844810 7598 consensus_queue.cc:260] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.844880 7598 ts_tablet_manager.cc:1434] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.844944 7598 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: Bootstrap starting.
I20251209 03:04:46.845343 7598 tablet_bootstrap.cc:654] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:46.845949 7598 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: No bootstrap required, opened a new log
I20251209 03:04:46.845996 7598 ts_tablet_manager.cc:1403] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:46.846096 7598 raft_consensus.cc:359] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.846138 7598 raft_consensus.cc:385] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:46.846150 7598 raft_consensus.cc:740] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a093a649e84cbb90ce0739ec97951a, State: Initialized, Role: FOLLOWER
I20251209 03:04:46.846181 7598 consensus_queue.cc:260] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.846247 7598 ts_tablet_manager.cc:1434] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.852967 7607 raft_consensus.cc:493] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:46.853026 7607 raft_consensus.cc:515] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.853191 7607 leader_election.cc:290] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245), 89c1072203c242959f590180e345a328 (127.2.170.195:39919)
I20251209 03:04:46.855621 7476 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f10c10456ac4cb2a1abaa81da3f11cd" candidate_uuid: "d7a093a649e84cbb90ce0739ec97951a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" is_pre_election: true
I20251209 03:04:46.855635 7551 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f10c10456ac4cb2a1abaa81da3f11cd" candidate_uuid: "d7a093a649e84cbb90ce0739ec97951a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89c1072203c242959f590180e345a328" is_pre_election: true
I20251209 03:04:46.855691 7476 raft_consensus.cc:2468] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a093a649e84cbb90ce0739ec97951a in term 0.
I20251209 03:04:46.855702 7551 raft_consensus.cc:2468] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a093a649e84cbb90ce0739ec97951a in term 0.
I20251209 03:04:46.855822 7366 leader_election.cc:304] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7928ecfc0bab4a6e88b37cdb44b48d2e, d7a093a649e84cbb90ce0739ec97951a; no voters:
I20251209 03:04:46.855914 7607 raft_consensus.cc:2804] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:46.855954 7607 raft_consensus.cc:493] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:46.855976 7607 raft_consensus.cc:3060] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.856511 7607 raft_consensus.cc:515] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.856616 7607 leader_election.cc:290] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [CANDIDATE]: Term 1 election: Requested vote from peers 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245), 89c1072203c242959f590180e345a328 (127.2.170.195:39919)
I20251209 03:04:46.856739 7551 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f10c10456ac4cb2a1abaa81da3f11cd" candidate_uuid: "d7a093a649e84cbb90ce0739ec97951a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89c1072203c242959f590180e345a328"
I20251209 03:04:46.856784 7551 raft_consensus.cc:3060] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.857247 7551 raft_consensus.cc:2468] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a093a649e84cbb90ce0739ec97951a in term 1.
I20251209 03:04:46.857329 7476 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f10c10456ac4cb2a1abaa81da3f11cd" candidate_uuid: "d7a093a649e84cbb90ce0739ec97951a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e"
I20251209 03:04:46.857380 7476 raft_consensus.cc:3060] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.857409 7365 leader_election.cc:304] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89c1072203c242959f590180e345a328, d7a093a649e84cbb90ce0739ec97951a; no voters:
I20251209 03:04:46.857519 7607 raft_consensus.cc:2804] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:46.857684 7607 raft_consensus.cc:697] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Becoming Leader. State: Replica: d7a093a649e84cbb90ce0739ec97951a, State: Running, Role: LEADER
I20251209 03:04:46.857734 7607 consensus_queue.cc:237] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.858075 7476 raft_consensus.cc:2468] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a093a649e84cbb90ce0739ec97951a in term 1.
I20251209 03:04:46.858350 7296 catalog_manager.cc:5654] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a reported cstate change: term changed from 0 to 1, leader changed from <none> to d7a093a649e84cbb90ce0739ec97951a (127.2.170.193). New cstate: current_term: 1 leader_uuid: "d7a093a649e84cbb90ce0739ec97951a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:46.876346 7607 raft_consensus.cc:493] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:46.876420 7607 raft_consensus.cc:515] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.876524 7607 leader_election.cc:290] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245), 89c1072203c242959f590180e345a328 (127.2.170.195:39919)
I20251209 03:04:46.876713 7476 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "57e328b687194376b21c2b3790a1f309" candidate_uuid: "d7a093a649e84cbb90ce0739ec97951a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" is_pre_election: true
I20251209 03:04:46.876713 7551 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "57e328b687194376b21c2b3790a1f309" candidate_uuid: "d7a093a649e84cbb90ce0739ec97951a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89c1072203c242959f590180e345a328" is_pre_election: true
I20251209 03:04:46.876828 7476 raft_consensus.cc:2468] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a093a649e84cbb90ce0739ec97951a in term 0.
I20251209 03:04:46.876879 7551 raft_consensus.cc:2468] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a093a649e84cbb90ce0739ec97951a in term 0.
I20251209 03:04:46.876981 7366 leader_election.cc:304] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7928ecfc0bab4a6e88b37cdb44b48d2e, d7a093a649e84cbb90ce0739ec97951a; no voters:
I20251209 03:04:46.877080 7607 raft_consensus.cc:2804] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:46.877127 7607 raft_consensus.cc:493] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:46.877152 7607 raft_consensus.cc:3060] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.877754 7607 raft_consensus.cc:515] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.877880 7607 leader_election.cc:290] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [CANDIDATE]: Term 1 election: Requested vote from peers 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245), 89c1072203c242959f590180e345a328 (127.2.170.195:39919)
I20251209 03:04:46.878000 7476 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "57e328b687194376b21c2b3790a1f309" candidate_uuid: "d7a093a649e84cbb90ce0739ec97951a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e"
I20251209 03:04:46.878013 7551 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "57e328b687194376b21c2b3790a1f309" candidate_uuid: "d7a093a649e84cbb90ce0739ec97951a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89c1072203c242959f590180e345a328"
I20251209 03:04:46.878069 7476 raft_consensus.cc:3060] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.878083 7551 raft_consensus.cc:3060] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.878708 7551 raft_consensus.cc:2468] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a093a649e84cbb90ce0739ec97951a in term 1.
I20251209 03:04:46.878708 7476 raft_consensus.cc:2468] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a093a649e84cbb90ce0739ec97951a in term 1.
I20251209 03:04:46.878875 7365 leader_election.cc:304] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89c1072203c242959f590180e345a328, d7a093a649e84cbb90ce0739ec97951a; no voters:
I20251209 03:04:46.878963 7607 raft_consensus.cc:2804] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:46.879012 7607 raft_consensus.cc:697] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Becoming Leader. State: Replica: d7a093a649e84cbb90ce0739ec97951a, State: Running, Role: LEADER
I20251209 03:04:46.879107 7607 consensus_queue.cc:237] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:46.879666 7296 catalog_manager.cc:5654] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a reported cstate change: term changed from 0 to 1, leader changed from <none> to d7a093a649e84cbb90ce0739ec97951a (127.2.170.193). New cstate: current_term: 1 leader_uuid: "d7a093a649e84cbb90ce0739ec97951a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:46.884187 7604 raft_consensus.cc:493] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:46.884243 7604 raft_consensus.cc:515] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.884375 7604 leader_election.cc:290] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7a093a649e84cbb90ce0739ec97951a (127.2.170.193:32885), 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245)
I20251209 03:04:46.886785 7476 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75ce916182974eb29ee2dac0d1c64068" candidate_uuid: "89c1072203c242959f590180e345a328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" is_pre_election: true
I20251209 03:04:46.886847 7400 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75ce916182974eb29ee2dac0d1c64068" candidate_uuid: "89c1072203c242959f590180e345a328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a093a649e84cbb90ce0739ec97951a" is_pre_election: true
I20251209 03:04:46.886868 7476 raft_consensus.cc:2468] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89c1072203c242959f590180e345a328 in term 0.
I20251209 03:04:46.886896 7400 raft_consensus.cc:2468] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89c1072203c242959f590180e345a328 in term 0.
I20251209 03:04:46.887032 7514 leader_election.cc:304] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89c1072203c242959f590180e345a328, d7a093a649e84cbb90ce0739ec97951a; no voters:
I20251209 03:04:46.887117 7604 raft_consensus.cc:2804] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:46.887149 7604 raft_consensus.cc:493] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:46.887162 7604 raft_consensus.cc:3060] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.887708 7604 raft_consensus.cc:515] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.887816 7604 leader_election.cc:290] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [CANDIDATE]: Term 1 election: Requested vote from peers d7a093a649e84cbb90ce0739ec97951a (127.2.170.193:32885), 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245)
I20251209 03:04:46.887931 7400 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75ce916182974eb29ee2dac0d1c64068" candidate_uuid: "89c1072203c242959f590180e345a328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a093a649e84cbb90ce0739ec97951a"
I20251209 03:04:46.887984 7400 raft_consensus.cc:3060] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.888463 7400 raft_consensus.cc:2468] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89c1072203c242959f590180e345a328 in term 1.
I20251209 03:04:46.888543 7476 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "75ce916182974eb29ee2dac0d1c64068" candidate_uuid: "89c1072203c242959f590180e345a328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e"
I20251209 03:04:46.888597 7476 raft_consensus.cc:3060] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:46.888610 7514 leader_election.cc:304] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89c1072203c242959f590180e345a328, d7a093a649e84cbb90ce0739ec97951a; no voters:
I20251209 03:04:46.888688 7604 raft_consensus.cc:2804] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:46.888803 7604 raft_consensus.cc:697] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 1 LEADER]: Becoming Leader. State: Replica: 89c1072203c242959f590180e345a328, State: Running, Role: LEADER
I20251209 03:04:46.888868 7604 consensus_queue.cc:237] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } }
I20251209 03:04:46.889180 7476 raft_consensus.cc:2468] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89c1072203c242959f590180e345a328 in term 1.
I20251209 03:04:46.889319 7296 catalog_manager.cc:5654] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89c1072203c242959f590180e345a328 (127.2.170.195). New cstate: current_term: 1 leader_uuid: "89c1072203c242959f590180e345a328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:46.890522 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:46.891683 7621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:46.891770 2731 server_base.cc:1047] running on GCE node
W20251209 03:04:46.891927 7622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:46.891980 7624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:46.892107 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:46.892138 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:46.892156 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249486892156 us; error 0 us; skew 500 ppm
I20251209 03:04:46.892804 2731 webserver.cc:492] Webserver started at http://127.2.170.196:34089/ using document root <none> and password file <none>
I20251209 03:04:46.892889 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:46.892931 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:46.892975 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:46.893218 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-3-root/instance:
uuid: "0b6580b2453945f89ec1c9580a595e0b"
format_stamp: "Formatted at 2025-12-09 03:04:46 on dist-test-slave-4cb0"
I20251209 03:04:46.894424 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:46.894979 7629 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.895190 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.895244 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-3-root
uuid: "0b6580b2453945f89ec1c9580a595e0b"
format_stamp: "Formatted at 2025-12-09 03:04:46 on dist-test-slave-4cb0"
I20251209 03:04:46.895283 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:46.941556 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:46.941834 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:46.941998 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:46.942203 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:46.942231 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.942260 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:46.942281 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:46.945307 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.196:36697
I20251209 03:04:46.945396 7692 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.196:36697 every 8 connection(s)
I20251209 03:04:46.948357 7693 heartbeater.cc:344] Connected to a master server at 127.2.170.254:46019
I20251209 03:04:46.948426 7693 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:46.948530 7693 heartbeater.cc:507] Master 127.2.170.254:46019 requested a full tablet report, sending...
I20251209 03:04:46.948740 7296 ts_manager.cc:194] Registered new tserver with Master: 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196:36697)
I20251209 03:04:46.948978 7607 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:46.949188 7296 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38580
W20251209 03:04:46.949925 7614 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20251209 03:04:46.949968 7614 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20251209 03:04:46.955647 7607 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:46.965234 7607 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:46.968122 7604 consensus_queue.cc:1048] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:46.974994 7604 consensus_queue.cc:1048] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:04:46.987920 7607 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251209 03:04:47.002650 7427 tablet.cc:2378] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251209 03:04:47.722087 7352 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:47.725124 7399 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "6f10c10456ac4cb2a1abaa81da3f11cd"
dest_uuid: "d7a093a649e84cbb90ce0739ec97951a"
mode: GRACEFUL
new_leader_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e"
from {username='slave'} at 127.0.0.1:33522
I20251209 03:04:47.725138 7400 consensus_queue.cc:237] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } }
I20251209 03:04:47.725217 7399 raft_consensus.cc:606] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Received request to transfer leadership to TS 7928ecfc0bab4a6e88b37cdb44b48d2e
I20251209 03:04:47.725378 7352 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20251209 03:04:47.725440 7352 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:47.725723 7551 raft_consensus.cc:1275] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Refusing update from remote peer d7a093a649e84cbb90ce0739ec97951a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:47.725883 7476 raft_consensus.cc:1275] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Refusing update from remote peer d7a093a649e84cbb90ce0739ec97951a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:04:47.726043 7701 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:04:47.726161 7712 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:04:47.726812 7551 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } } }
I20251209 03:04:47.726819 7476 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } } }
I20251209 03:04:47.727999 7296 catalog_manager.cc:5654] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e reported cstate change: config changed from index -1 to 2, NON_VOTER 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "d7a093a649e84cbb90ce0739ec97951a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } } }
W20251209 03:04:47.728892 7366 consensus_peers.cc:597] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a -> Peer 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196:36697): Couldn't send request to peer 0b6580b2453945f89ec1c9580a595e0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 57e328b687194376b21c2b3790a1f309. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:47.750795 7712 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } } }
W20251209 03:04:47.752447 7346 auto_rebalancer.cc:253] failed to send replica move request: Remote error: Service unavailable: leader transfer in progress
W20251209 03:04:47.753450 7346 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 6f10c10456ac4cb2a1abaa81da3f11cd, TS 89c1072203c242959f590180e345a328 -> TS 0b6580b2453945f89ec1c9580a595e0b move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
W20251209 03:04:47.753506 7346 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 6f10c10456ac4cb2a1abaa81da3f11cd, TS 89c1072203c242959f590180e345a328 -> TS 0b6580b2453945f89ec1c9580a595e0b move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:47.829567 7720 ts_tablet_manager.cc:933] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: Initiating tablet copy from peer d7a093a649e84cbb90ce0739ec97951a (127.2.170.193:32885)
I20251209 03:04:47.829806 7720 tablet_copy_client.cc:323] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.193:32885
I20251209 03:04:47.833209 7410 tablet_copy_service.cc:140] P d7a093a649e84cbb90ce0739ec97951a: Received BeginTabletCopySession request for tablet 57e328b687194376b21c2b3790a1f309 from peer 0b6580b2453945f89ec1c9580a595e0b ({username='slave'} at 127.0.0.1:33532)
I20251209 03:04:47.833292 7410 tablet_copy_service.cc:161] P d7a093a649e84cbb90ce0739ec97951a: Beginning new tablet copy session on tablet 57e328b687194376b21c2b3790a1f309 from peer 0b6580b2453945f89ec1c9580a595e0b at {username='slave'} at 127.0.0.1:33532: session id = 0b6580b2453945f89ec1c9580a595e0b-57e328b687194376b21c2b3790a1f309
I20251209 03:04:47.833899 7410 tablet_copy_source_session.cc:215] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:47.834234 7720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57e328b687194376b21c2b3790a1f309. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:47.835651 7720 tablet_copy_client.cc:806] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:47.835777 7720 tablet_copy_client.cc:670] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:47.836385 7720 tablet_copy_client.cc:538] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:47.837060 7720 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: Bootstrap starting.
I20251209 03:04:47.837790 7614 raft_consensus.cc:993] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a: : Instructing follower 7928ecfc0bab4a6e88b37cdb44b48d2e to start an election
I20251209 03:04:47.837881 7712 raft_consensus.cc:1081] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Signalling peer 7928ecfc0bab4a6e88b37cdb44b48d2e to start an election
I20251209 03:04:47.838202 7476 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "6f10c10456ac4cb2a1abaa81da3f11cd"
dest_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e"
from {username='slave'} at 127.0.0.1:50606
I20251209 03:04:47.838274 7476 raft_consensus.cc:493] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251209 03:04:47.838306 7476 raft_consensus.cc:3060] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:47.838621 7720 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:47.838775 7720 tablet_bootstrap.cc:492] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: Bootstrap complete.
I20251209 03:04:47.838891 7720 ts_tablet_manager.cc:1403] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20251209 03:04:47.838963 7476 raft_consensus.cc:515] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:47.839006 7720 raft_consensus.cc:359] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } }
I20251209 03:04:47.839071 7720 raft_consensus.cc:740] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0b6580b2453945f89ec1c9580a595e0b, State: Initialized, Role: LEARNER
I20251209 03:04:47.839098 7720 consensus_queue.cc:260] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } }
I20251209 03:04:47.839133 7476 leader_election.cc:290] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [CANDIDATE]: Term 2 election: Requested vote from peers d7a093a649e84cbb90ce0739ec97951a (127.2.170.193:32885), 89c1072203c242959f590180e345a328 (127.2.170.195:39919)
I20251209 03:04:47.839252 7720 ts_tablet_manager.cc:1434] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251209 03:04:47.839421 7693 heartbeater.cc:499] Master 127.2.170.254:46019 was elected leader, sending a full tablet report...
I20251209 03:04:47.839910 7410 tablet_copy_service.cc:342] P d7a093a649e84cbb90ce0739ec97951a: Request end of tablet copy session 0b6580b2453945f89ec1c9580a595e0b-57e328b687194376b21c2b3790a1f309 received from {username='slave'} at 127.0.0.1:33532
I20251209 03:04:47.839955 7410 tablet_copy_service.cc:434] P d7a093a649e84cbb90ce0739ec97951a: ending tablet copy session 0b6580b2453945f89ec1c9580a595e0b-57e328b687194376b21c2b3790a1f309 on tablet 57e328b687194376b21c2b3790a1f309 with peer 0b6580b2453945f89ec1c9580a595e0b
I20251209 03:04:47.842617 7399 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f10c10456ac4cb2a1abaa81da3f11cd" candidate_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7a093a649e84cbb90ce0739ec97951a"
I20251209 03:04:47.842718 7399 raft_consensus.cc:3055] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Stepping down as leader of term 1
I20251209 03:04:47.842751 7399 raft_consensus.cc:740] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d7a093a649e84cbb90ce0739ec97951a, State: Running, Role: LEADER
I20251209 03:04:47.842810 7399 consensus_queue.cc:260] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:47.842892 7399 raft_consensus.cc:3060] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:47.843067 7551 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f10c10456ac4cb2a1abaa81da3f11cd" candidate_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "89c1072203c242959f590180e345a328"
I20251209 03:04:47.843127 7551 raft_consensus.cc:3060] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:47.843585 7551 raft_consensus.cc:2468] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7928ecfc0bab4a6e88b37cdb44b48d2e in term 2.
I20251209 03:04:47.843614 7399 raft_consensus.cc:2468] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7928ecfc0bab4a6e88b37cdb44b48d2e in term 2.
I20251209 03:04:47.843798 7441 leader_election.cc:304] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7928ecfc0bab4a6e88b37cdb44b48d2e, 89c1072203c242959f590180e345a328; no voters:
I20251209 03:04:47.843917 7715 raft_consensus.cc:2804] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:47.844023 7715 raft_consensus.cc:697] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 2 LEADER]: Becoming Leader. State: Replica: 7928ecfc0bab4a6e88b37cdb44b48d2e, State: Running, Role: LEADER
I20251209 03:04:47.844075 7715 consensus_queue.cc:237] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } }
I20251209 03:04:47.844471 7297 catalog_manager.cc:5654] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e reported cstate change: term changed from 1 to 2, leader changed from d7a093a649e84cbb90ce0739ec97951a (127.2.170.193) to 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194). New cstate: current_term: 2 leader_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:47.923720 7667 raft_consensus.cc:1217] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251209 03:04:47.943668 7399 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251209 03:04:47.943883 7715 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:04:47.963955 7551 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 2 FOLLOWER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251209 03:04:47.964216 7731 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:04:48.134999 7719 raft_consensus.cc:1064] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: attempting to promote NON_VOTER 0b6580b2453945f89ec1c9580a595e0b to VOTER
I20251209 03:04:48.135232 7719 consensus_queue.cc:237] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } }
I20251209 03:04:48.135668 7551 raft_consensus.cc:1275] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Refusing update from remote peer d7a093a649e84cbb90ce0739ec97951a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:04:48.135713 7476 raft_consensus.cc:1275] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Refusing update from remote peer d7a093a649e84cbb90ce0739ec97951a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:04:48.135713 7667 raft_consensus.cc:1275] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 LEARNER]: Refusing update from remote peer d7a093a649e84cbb90ce0739ec97951a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:04:48.135916 7712 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:04:48.136044 7614 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:04:48.136155 7712 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:04:48.136777 7551 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.136901 7667 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.137957 7297 catalog_manager.cc:5654] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 reported cstate change: config changed from index 2 to 3, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d7a093a649e84cbb90ce0739ec97951a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.198800 7614 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.200055 7476 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } attrs { replace: true } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.200486 7400 consensus_queue.cc:237] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } }
I20251209 03:04:48.200896 7551 raft_consensus.cc:1275] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Refusing update from remote peer d7a093a649e84cbb90ce0739ec97951a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:04:48.200930 7667 raft_consensus.cc:1275] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 FOLLOWER]: Refusing update from remote peer d7a093a649e84cbb90ce0739ec97951a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:04:48.201076 7735 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:04:48.201188 7719 consensus_queue.cc:1048] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:04:48.201826 7551 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.202651 7667 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.202972 7296 catalog_manager.cc:5654] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 reported cstate change: config changed from index 3 to 4, VOTER 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194) evicted. New cstate: current_term: 1 leader_uuid: "d7a093a649e84cbb90ce0739ec97951a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.235857 7466 tablet_service.cc:1552] Processing DeleteTablet for tablet 57e328b687194376b21c2b3790a1f309 with delete_type TABLET_DATA_TOMBSTONED (TS 7928ecfc0bab4a6e88b37cdb44b48d2e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50592
I20251209 03:04:48.236109 7736 tablet_replica.cc:333] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: stopping tablet replica
I20251209 03:04:48.236196 7736 raft_consensus.cc:2243] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:48.236263 7736 raft_consensus.cc:2272] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:48.236539 7736 ts_tablet_manager.cc:1916] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:48.238237 7736 ts_tablet_manager.cc:1929] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20251209 03:04:48.238301 7736 log.cc:1199] T 57e328b687194376b21c2b3790a1f309 P 7928ecfc0bab4a6e88b37cdb44b48d2e: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/57e328b687194376b21c2b3790a1f309
I20251209 03:04:48.238658 7284 catalog_manager.cc:4985] TS 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245): tablet 57e328b687194376b21c2b3790a1f309 (table test-workload [id=f6c0eeda38cd415996e49d3429315fea]) successfully deleted
I20251209 03:04:48.370383 7719 raft_consensus.cc:2955] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:48.371363 7281 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 57e328b687194376b21c2b3790a1f309 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:49.378366 7476 consensus_queue.cc:237] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } }
I20251209 03:04:49.378993 7551 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 2 FOLLOWER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251209 03:04:49.378988 7399 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251209 03:04:49.379235 7740 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:04:49.379366 7741 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:04:49.380218 7399 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } } }
I20251209 03:04:49.380228 7551 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } } }
I20251209 03:04:49.381247 7296 catalog_manager.cc:5654] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 reported cstate change: config changed from index -1 to 3, NON_VOTER 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) added. New cstate: current_term: 2 leader_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } } }
W20251209 03:04:49.382062 7442 consensus_peers.cc:597] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e -> Peer 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196:36697): Couldn't send request to peer 0b6580b2453945f89ec1c9580a595e0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f10c10456ac4cb2a1abaa81da3f11cd. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:04:49.409190 7741 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } } }
I20251209 03:04:49.473374 7751 ts_tablet_manager.cc:933] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: Initiating tablet copy from peer 7928ecfc0bab4a6e88b37cdb44b48d2e (127.2.170.194:44245)
I20251209 03:04:49.473603 7751 tablet_copy_client.cc:323] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:44245
I20251209 03:04:49.476763 7486 tablet_copy_service.cc:140] P 7928ecfc0bab4a6e88b37cdb44b48d2e: Received BeginTabletCopySession request for tablet 6f10c10456ac4cb2a1abaa81da3f11cd from peer 0b6580b2453945f89ec1c9580a595e0b ({username='slave'} at 127.0.0.1:33940)
I20251209 03:04:49.476837 7486 tablet_copy_service.cc:161] P 7928ecfc0bab4a6e88b37cdb44b48d2e: Beginning new tablet copy session on tablet 6f10c10456ac4cb2a1abaa81da3f11cd from peer 0b6580b2453945f89ec1c9580a595e0b at {username='slave'} at 127.0.0.1:33940: session id = 0b6580b2453945f89ec1c9580a595e0b-6f10c10456ac4cb2a1abaa81da3f11cd
I20251209 03:04:49.477285 7486 tablet_copy_source_session.cc:215] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:04:49.477543 7751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f10c10456ac4cb2a1abaa81da3f11cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:49.478976 7751 tablet_copy_client.cc:806] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: tablet copy: Starting download of 0 data blocks...
I20251209 03:04:49.479137 7751 tablet_copy_client.cc:670] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: tablet copy: Starting download of 1 WAL segments...
I20251209 03:04:49.479648 7751 tablet_copy_client.cc:538] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:04:49.480324 7751 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: Bootstrap starting.
I20251209 03:04:49.481482 7751 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:04:49.481652 7751 tablet_bootstrap.cc:492] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: Bootstrap complete.
I20251209 03:04:49.481761 7751 ts_tablet_manager.cc:1403] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.002s
I20251209 03:04:49.481916 7751 raft_consensus.cc:359] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } }
I20251209 03:04:49.481976 7751 raft_consensus.cc:740] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 0b6580b2453945f89ec1c9580a595e0b, State: Initialized, Role: LEARNER
I20251209 03:04:49.482007 7751 consensus_queue.cc:260] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: true } }
I20251209 03:04:49.482213 7751 ts_tablet_manager.cc:1434] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:49.482358 7486 tablet_copy_service.cc:342] P 7928ecfc0bab4a6e88b37cdb44b48d2e: Request end of tablet copy session 0b6580b2453945f89ec1c9580a595e0b-6f10c10456ac4cb2a1abaa81da3f11cd received from {username='slave'} at 127.0.0.1:33940
I20251209 03:04:49.482412 7486 tablet_copy_service.cc:434] P 7928ecfc0bab4a6e88b37cdb44b48d2e: ending tablet copy session 0b6580b2453945f89ec1c9580a595e0b-6f10c10456ac4cb2a1abaa81da3f11cd on tablet 6f10c10456ac4cb2a1abaa81da3f11cd with peer 0b6580b2453945f89ec1c9580a595e0b
I20251209 03:04:49.549515 7667 raft_consensus.cc:1217] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251209 03:04:49.676868 7741 raft_consensus.cc:1064] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e: attempting to promote NON_VOTER 0b6580b2453945f89ec1c9580a595e0b to VOTER
I20251209 03:04:49.677220 7741 consensus_queue.cc:237] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } }
I20251209 03:04:49.677713 7399 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251209 03:04:49.677870 7667 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 LEARNER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251209 03:04:49.678052 7740 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:04:49.677718 7551 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 2 FOLLOWER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251209 03:04:49.678210 7715 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:04:49.678293 7740 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:04:49.678905 7551 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.679417 7667 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.680048 7297 catalog_manager.cc:5654] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 reported cstate change: config changed from index 3 to 4, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.725557 7352 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:49.725728 7352 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:49.725790 7352 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:49.739887 7740 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.741068 7399 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 0b6580b2453945f89ec1c9580a595e0b (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "89c1072203c242959f590180e345a328" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 39919 } attrs { replace: true } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.761641 7476 consensus_queue.cc:237] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } }
I20251209 03:04:49.762116 7399 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251209 03:04:49.762116 7667 raft_consensus.cc:1275] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 FOLLOWER]: Refusing update from remote peer 7928ecfc0bab4a6e88b37cdb44b48d2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251209 03:04:49.762332 7741 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:04:49.762428 7740 consensus_queue.cc:1048] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:04:49.763254 7667 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 89c1072203c242959f590180e345a328 (127.2.170.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.763288 7399 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 89c1072203c242959f590180e345a328 (127.2.170.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.764426 7295 catalog_manager.cc:5654] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b reported cstate change: config changed from index 4 to 5, VOTER 89c1072203c242959f590180e345a328 (127.2.170.195) evicted. New cstate: current_term: 2 leader_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.771276 7541 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f10c10456ac4cb2a1abaa81da3f11cd with delete_type TABLET_DATA_TOMBSTONED (TS 89c1072203c242959f590180e345a328 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52898
I20251209 03:04:49.771540 7758 tablet_replica.cc:333] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: stopping tablet replica
I20251209 03:04:49.771605 7758 raft_consensus.cc:2243] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:49.771667 7758 raft_consensus.cc:2272] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:49.771909 7758 ts_tablet_manager.cc:1916] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:04:49.773348 7758 ts_tablet_manager.cc:1929] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20251209 03:04:49.773403 7758 log.cc:1199] T 6f10c10456ac4cb2a1abaa81da3f11cd P 89c1072203c242959f590180e345a328: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.AutoRebalancingUnstableCluster.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/6f10c10456ac4cb2a1abaa81da3f11cd
I20251209 03:04:49.773742 7283 catalog_manager.cc:4985] TS 89c1072203c242959f590180e345a328 (127.2.170.195:39919): tablet 6f10c10456ac4cb2a1abaa81da3f11cd (table test-workload [id=f6c0eeda38cd415996e49d3429315fea]) successfully deleted
I20251209 03:04:49.880457 7757 raft_consensus.cc:2955] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 89c1072203c242959f590180e345a328 (127.2.170.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d7a093a649e84cbb90ce0739ec97951a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 32885 } } peers { permanent_uuid: "7928ecfc0bab4a6e88b37cdb44b48d2e" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44245 } } peers { permanent_uuid: "0b6580b2453945f89ec1c9580a595e0b" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 36697 } attrs { promote: false } } }
I20251209 03:04:49.881587 7284 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 6f10c10456ac4cb2a1abaa81da3f11cd with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:04:51.725929 7352 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:51.726080 7352 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:51.726119 7352 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:53.726301 7352 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:53.726480 7352 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:53.726529 7352 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:55.726723 7352 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:55.726886 7352 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:55.726917 7352 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:55.995919 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:04:55.998715 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:55.998946 2731 tablet_replica.cc:333] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a: stopping tablet replica
I20251209 03:04:55.999010 2731 raft_consensus.cc:2243] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:04:55.999081 2731 raft_consensus.cc:2272] T 57e328b687194376b21c2b3790a1f309 P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:55.999249 2731 tablet_replica.cc:333] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a: stopping tablet replica
I20251209 03:04:55.999289 2731 raft_consensus.cc:2243] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:55.999320 2731 raft_consensus.cc:2272] T 75ce916182974eb29ee2dac0d1c64068 P d7a093a649e84cbb90ce0739ec97951a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:55.999426 2731 tablet_replica.cc:333] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a: stopping tablet replica
I20251209 03:04:55.999457 2731 raft_consensus.cc:2243] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:55.999508 2731 raft_consensus.cc:2272] T 6f10c10456ac4cb2a1abaa81da3f11cd P d7a093a649e84cbb90ce0739ec97951a [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:56.011607 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:04:56.012694 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:04:56.015609 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:56.015727 2731 tablet_replica.cc:333] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e: stopping tablet replica
I20251209 03:04:56.015796 2731 raft_consensus.cc:2243] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:56.015832 2731 raft_consensus.cc:2272] T 75ce916182974eb29ee2dac0d1c64068 P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:56.015957 2731 tablet_replica.cc:333] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e: stopping tablet replica
I20251209 03:04:56.015990 2731 raft_consensus.cc:2243] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:04:56.016032 2731 raft_consensus.cc:2272] T 6f10c10456ac4cb2a1abaa81da3f11cd P 7928ecfc0bab4a6e88b37cdb44b48d2e [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:56.028271 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:04:56.029220 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:04:56.032068 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:56.032168 2731 tablet_replica.cc:333] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328: stopping tablet replica
I20251209 03:04:56.032231 2731 raft_consensus.cc:2243] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:04:56.032295 2731 raft_consensus.cc:2272] T 75ce916182974eb29ee2dac0d1c64068 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:56.032423 2731 tablet_replica.cc:333] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328: stopping tablet replica
I20251209 03:04:56.032450 2731 raft_consensus.cc:2243] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:56.032485 2731 raft_consensus.cc:2272] T 57e328b687194376b21c2b3790a1f309 P 89c1072203c242959f590180e345a328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:56.043984 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:04:56.044915 2731 tablet_server.cc:178] TabletServer@127.2.170.196:0 shutting down...
I20251209 03:04:56.047520 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:56.047616 2731 tablet_replica.cc:333] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b: stopping tablet replica
I20251209 03:04:56.047670 2731 raft_consensus.cc:2243] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:56.047727 2731 raft_consensus.cc:2272] T 6f10c10456ac4cb2a1abaa81da3f11cd P 0b6580b2453945f89ec1c9580a595e0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:56.047874 2731 tablet_replica.cc:333] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b: stopping tablet replica
I20251209 03:04:56.047911 2731 raft_consensus.cc:2243] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:04:56.047953 2731 raft_consensus.cc:2272] T 57e328b687194376b21c2b3790a1f309 P 0b6580b2453945f89ec1c9580a595e0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:56.059314 2731 tablet_server.cc:195] TabletServer@127.2.170.196:0 shutdown complete.
I20251209 03:04:56.060220 2731 master.cc:561] Master@127.2.170.254:46019 shutting down...
I20251209 03:04:56.062655 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:04:56.062739 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:56.062772 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7cb7da8637bd49748adf681e73ff1ae4: stopping tablet replica
I20251209 03:04:56.074291 2731 master.cc:583] Master@127.2.170.254:46019 shutdown complete.
[ OK ] AutoRebalancerTest.AutoRebalancingUnstableCluster (10397 ms)
[ RUN ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor
I20251209 03:04:56.079799 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:41219
I20251209 03:04:56.079938 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:56.081120 7761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:56.081231 2731 server_base.cc:1047] running on GCE node
W20251209 03:04:56.081156 7762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:56.081324 7764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:56.081471 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:56.081509 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:56.081521 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249496081521 us; error 0 us; skew 500 ppm
I20251209 03:04:56.082067 2731 webserver.cc:492] Webserver started at http://127.2.170.254:34457/ using document root <none> and password file <none>
I20251209 03:04:56.082129 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:56.082170 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:56.082211 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:56.082440 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "ad8eb869542e407193df6f3d312b89e5"
format_stamp: "Formatted at 2025-12-09 03:04:56 on dist-test-slave-4cb0"
I20251209 03:04:56.083204 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:56.083575 7769 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.083696 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.083748 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "ad8eb869542e407193df6f3d312b89e5"
format_stamp: "Formatted at 2025-12-09 03:04:56 on dist-test-slave-4cb0"
I20251209 03:04:56.083786 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:56.087191 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:56.087320 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:56.089514 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:41219
I20251209 03:04:56.092167 7821 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:41219 every 8 connection(s)
I20251209 03:04:56.092263 7822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:56.093170 7822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5: Bootstrap starting.
I20251209 03:04:56.093416 7822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:56.093915 7822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5: No bootstrap required, opened a new log
I20251209 03:04:56.094029 7822 raft_consensus.cc:359] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8eb869542e407193df6f3d312b89e5" member_type: VOTER }
I20251209 03:04:56.094072 7822 raft_consensus.cc:385] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:56.094084 7822 raft_consensus.cc:740] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8eb869542e407193df6f3d312b89e5, State: Initialized, Role: FOLLOWER
I20251209 03:04:56.094125 7822 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8eb869542e407193df6f3d312b89e5" member_type: VOTER }
I20251209 03:04:56.094158 7822 raft_consensus.cc:399] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:04:56.094177 7822 raft_consensus.cc:493] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:04:56.094197 7822 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:56.094615 7822 raft_consensus.cc:515] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8eb869542e407193df6f3d312b89e5" member_type: VOTER }
I20251209 03:04:56.094667 7822 leader_election.cc:304] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad8eb869542e407193df6f3d312b89e5; no voters:
I20251209 03:04:56.094743 7822 leader_election.cc:290] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:04:56.094805 7825 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:56.094861 7822 sys_catalog.cc:565] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:04:56.094940 7825 raft_consensus.cc:697] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 1 LEADER]: Becoming Leader. State: Replica: ad8eb869542e407193df6f3d312b89e5, State: Running, Role: LEADER
I20251209 03:04:56.095016 7825 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8eb869542e407193df6f3d312b89e5" member_type: VOTER }
I20251209 03:04:56.095206 7826 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad8eb869542e407193df6f3d312b89e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8eb869542e407193df6f3d312b89e5" member_type: VOTER } }
I20251209 03:04:56.095256 7826 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [sys.catalog]: This master's current role is: LEADER
I20251209 03:04:56.095376 7827 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad8eb869542e407193df6f3d312b89e5. Latest consensus state: current_term: 1 leader_uuid: "ad8eb869542e407193df6f3d312b89e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8eb869542e407193df6f3d312b89e5" member_type: VOTER } }
I20251209 03:04:56.095417 7831 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:04:56.095427 7827 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [sys.catalog]: This master's current role is: LEADER
I20251209 03:04:56.095638 7831 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:04:56.096071 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:04:56.096201 7831 catalog_manager.cc:1357] Generated new cluster ID: bd8b94f42a044feeb372a539491e4d3d
I20251209 03:04:56.096246 7831 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:04:56.105631 7831 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:04:56.106105 7831 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:04:56.111802 7831 catalog_manager.cc:6027] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5: Generated new TSK 0
I20251209 03:04:56.111882 7831 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:04:56.127907 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:56.129347 7843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:56.129357 7844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:56.129431 7846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:56.129602 2731 server_base.cc:1047] running on GCE node
I20251209 03:04:56.129750 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:56.129781 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:56.129792 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249496129792 us; error 0 us; skew 500 ppm
I20251209 03:04:56.130352 2731 webserver.cc:492] Webserver started at http://127.2.170.193:33561/ using document root <none> and password file <none>
I20251209 03:04:56.130432 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:56.130475 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:56.130522 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:56.130795 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "7fcd15804e074edb93a78daa879c498f"
format_stamp: "Formatted at 2025-12-09 03:04:56 on dist-test-slave-4cb0"
I20251209 03:04:56.131606 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:04:56.132019 7851 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.132149 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.132189 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "7fcd15804e074edb93a78daa879c498f"
format_stamp: "Formatted at 2025-12-09 03:04:56 on dist-test-slave-4cb0"
I20251209 03:04:56.132231 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:56.157254 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:56.157488 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:56.157636 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:56.157896 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:56.157925 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.157955 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:56.157972 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.160609 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:41191
I20251209 03:04:56.160666 7914 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:41191 every 8 connection(s)
I20251209 03:04:56.160988 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:56.162252 7920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:56.162259 7919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:56.162542 2731 server_base.cc:1047] running on GCE node
W20251209 03:04:56.162659 7922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:56.162849 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:56.162889 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:56.162905 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249496162905 us; error 0 us; skew 500 ppm
I20251209 03:04:56.163559 2731 webserver.cc:492] Webserver started at http://127.2.170.194:43153/ using document root <none> and password file <none>
I20251209 03:04:56.163695 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:56.163753 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:56.163796 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:56.164136 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "ea7dc5c967df4885abae3266d546830a"
format_stamp: "Formatted at 2025-12-09 03:04:56 on dist-test-slave-4cb0"
I20251209 03:04:56.165217 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:56.165701 7927 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.165838 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.165884 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "ea7dc5c967df4885abae3266d546830a"
format_stamp: "Formatted at 2025-12-09 03:04:56 on dist-test-slave-4cb0"
I20251209 03:04:56.165936 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:56.168632 7915 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41219
I20251209 03:04:56.168709 7915 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:56.168824 7915 heartbeater.cc:507] Master 127.2.170.254:41219 requested a full tablet report, sending...
I20251209 03:04:56.169071 7786 ts_manager.cc:194] Registered new tserver with Master: 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191)
I20251209 03:04:56.169596 7786 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41792
I20251209 03:04:56.177507 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:56.177824 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:56.177996 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:56.178190 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:56.178216 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.178248 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:56.178272 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.180825 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:39995
I20251209 03:04:56.180893 7990 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:39995 every 8 connection(s)
I20251209 03:04:56.181195 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:04:56.182297 7995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:56.182390 7994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:04:56.182547 7997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:04:56.182858 2731 server_base.cc:1047] running on GCE node
I20251209 03:04:56.182994 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:04:56.183027 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:04:56.183051 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249496183051 us; error 0 us; skew 500 ppm
I20251209 03:04:56.183642 2731 webserver.cc:492] Webserver started at http://127.2.170.195:42929/ using document root <none> and password file <none>
I20251209 03:04:56.183710 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:04:56.183755 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:04:56.183802 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:04:56.184149 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "c3b2dd07c0754f6d8fed1907f0e09045"
format_stamp: "Formatted at 2025-12-09 03:04:56 on dist-test-slave-4cb0"
I20251209 03:04:56.184542 7991 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41219
I20251209 03:04:56.184600 7991 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:56.184702 7991 heartbeater.cc:507] Master 127.2.170.254:41219 requested a full tablet report, sending...
I20251209 03:04:56.184985 7786 ts_manager.cc:194] Registered new tserver with Master: ea7dc5c967df4885abae3266d546830a (127.2.170.194:39995)
I20251209 03:04:56.185333 7786 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41802
I20251209 03:04:56.185570 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:56.186157 8002 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.186336 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.186398 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "c3b2dd07c0754f6d8fed1907f0e09045"
format_stamp: "Formatted at 2025-12-09 03:04:56 on dist-test-slave-4cb0"
I20251209 03:04:56.186434 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:04:56.210271 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:04:56.210506 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:04:56.210657 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:04:56.210824 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:04:56.210850 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.210875 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:04:56.210896 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:56.214293 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:33787
I20251209 03:04:56.214355 8065 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:33787 every 8 connection(s)
I20251209 03:04:56.217414 8066 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41219
I20251209 03:04:56.217480 8066 heartbeater.cc:461] Registering TS with master...
I20251209 03:04:56.217586 8066 heartbeater.cc:507] Master 127.2.170.254:41219 requested a full tablet report, sending...
I20251209 03:04:56.217895 7786 ts_manager.cc:194] Registered new tserver with Master: c3b2dd07c0754f6d8fed1907f0e09045 (127.2.170.195:33787)
I20251209 03:04:56.218261 7786 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41812
I20251209 03:04:56.218896 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004419791s
I20251209 03:04:57.170186 7915 heartbeater.cc:499] Master 127.2.170.254:41219 was elected leader, sending a full tablet report...
I20251209 03:04:57.185917 7991 heartbeater.cc:499] Master 127.2.170.254:41219 was elected leader, sending a full tablet report...
I20251209 03:04:57.218775 8066 heartbeater.cc:499] Master 127.2.170.254:41219 was elected leader, sending a full tablet report...
I20251209 03:04:57.244065 2731 test_util.cc:276] Using random seed: 758718878
I20251209 03:04:57.249413 7786 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41816:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:04:57.249560 7786 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:04:57.254384 7955 tablet_service.cc:1505] Processing CreateTablet for tablet 02e29391c5464fd589524d831908bf32 (DEFAULT_TABLE table=test-workload [id=bf9195ee22b54628a4c6d4e84ee02483]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:04:57.254550 7955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e29391c5464fd589524d831908bf32. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:57.254837 7879 tablet_service.cc:1505] Processing CreateTablet for tablet 02e29391c5464fd589524d831908bf32 (DEFAULT_TABLE table=test-workload [id=bf9195ee22b54628a4c6d4e84ee02483]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:04:57.254837 8030 tablet_service.cc:1505] Processing CreateTablet for tablet 02e29391c5464fd589524d831908bf32 (DEFAULT_TABLE table=test-workload [id=bf9195ee22b54628a4c6d4e84ee02483]), partition=RANGE (key) PARTITION UNBOUNDED
I20251209 03:04:57.254984 7879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e29391c5464fd589524d831908bf32. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:57.254985 8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e29391c5464fd589524d831908bf32. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:04:57.255955 8086 tablet_bootstrap.cc:492] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a: Bootstrap starting.
I20251209 03:04:57.256343 8087 tablet_bootstrap.cc:492] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f: Bootstrap starting.
I20251209 03:04:57.256367 8086 tablet_bootstrap.cc:654] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:57.256457 8088 tablet_bootstrap.cc:492] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045: Bootstrap starting.
I20251209 03:04:57.256880 8087 tablet_bootstrap.cc:654] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:57.256932 8088 tablet_bootstrap.cc:654] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045: Neither blocks nor log segments found. Creating new log.
I20251209 03:04:57.257089 8086 tablet_bootstrap.cc:492] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a: No bootstrap required, opened a new log
I20251209 03:04:57.257164 8086 ts_tablet_manager.cc:1403] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:57.257360 8087 tablet_bootstrap.cc:492] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f: No bootstrap required, opened a new log
I20251209 03:04:57.257335 8086 raft_consensus.cc:359] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.257418 8087 ts_tablet_manager.cc:1403] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:57.257434 8086 raft_consensus.cc:385] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:57.257462 8086 raft_consensus.cc:740] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea7dc5c967df4885abae3266d546830a, State: Initialized, Role: FOLLOWER
I20251209 03:04:57.257507 8086 consensus_queue.cc:260] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.257566 8087 raft_consensus.cc:359] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.257611 8087 raft_consensus.cc:385] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:57.257651 8087 raft_consensus.cc:740] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fcd15804e074edb93a78daa879c498f, State: Initialized, Role: FOLLOWER
I20251209 03:04:57.257690 8086 ts_tablet_manager.cc:1434] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:57.257725 8087 consensus_queue.cc:260] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.257617 8088 tablet_bootstrap.cc:492] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045: No bootstrap required, opened a new log
I20251209 03:04:57.257844 8088 ts_tablet_manager.cc:1403] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:04:57.257860 8087 ts_tablet_manager.cc:1434] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:57.257984 8088 raft_consensus.cc:359] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.258037 8088 raft_consensus.cc:385] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:04:57.258055 8088 raft_consensus.cc:740] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b2dd07c0754f6d8fed1907f0e09045, State: Initialized, Role: FOLLOWER
I20251209 03:04:57.258111 8088 consensus_queue.cc:260] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.258352 8088 ts_tablet_manager.cc:1434] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:04:57.449468 8093 raft_consensus.cc:493] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:57.449574 8093 raft_consensus.cc:515] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.449769 8093 leader_election.cc:290] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea7dc5c967df4885abae3266d546830a (127.2.170.194:39995), c3b2dd07c0754f6d8fed1907f0e09045 (127.2.170.195:33787)
I20251209 03:04:57.453413 7965 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02e29391c5464fd589524d831908bf32" candidate_uuid: "7fcd15804e074edb93a78daa879c498f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7dc5c967df4885abae3266d546830a" is_pre_election: true
I20251209 03:04:57.453413 8040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02e29391c5464fd589524d831908bf32" candidate_uuid: "7fcd15804e074edb93a78daa879c498f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" is_pre_election: true
I20251209 03:04:57.453533 7965 raft_consensus.cc:2468] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fcd15804e074edb93a78daa879c498f in term 0.
I20251209 03:04:57.453550 8040 raft_consensus.cc:2468] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fcd15804e074edb93a78daa879c498f in term 0.
I20251209 03:04:57.453755 7854 leader_election.cc:304] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fcd15804e074edb93a78daa879c498f, ea7dc5c967df4885abae3266d546830a; no voters:
I20251209 03:04:57.453889 8093 raft_consensus.cc:2804] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:04:57.453945 8093 raft_consensus.cc:493] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:57.453972 8093 raft_consensus.cc:3060] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:57.454613 8093 raft_consensus.cc:515] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.454768 8093 leader_election.cc:290] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [CANDIDATE]: Term 1 election: Requested vote from peers ea7dc5c967df4885abae3266d546830a (127.2.170.194:39995), c3b2dd07c0754f6d8fed1907f0e09045 (127.2.170.195:33787)
I20251209 03:04:57.454972 8040 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02e29391c5464fd589524d831908bf32" candidate_uuid: "7fcd15804e074edb93a78daa879c498f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b2dd07c0754f6d8fed1907f0e09045"
I20251209 03:04:57.455032 8040 raft_consensus.cc:3060] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:57.455011 7965 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02e29391c5464fd589524d831908bf32" candidate_uuid: "7fcd15804e074edb93a78daa879c498f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7dc5c967df4885abae3266d546830a"
I20251209 03:04:57.455067 7965 raft_consensus.cc:3060] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:04:57.455687 8040 raft_consensus.cc:2468] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fcd15804e074edb93a78daa879c498f in term 1.
I20251209 03:04:57.455695 7965 raft_consensus.cc:2468] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fcd15804e074edb93a78daa879c498f in term 1.
I20251209 03:04:57.455874 7854 leader_election.cc:304] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fcd15804e074edb93a78daa879c498f, c3b2dd07c0754f6d8fed1907f0e09045; no voters:
I20251209 03:04:57.455957 8093 raft_consensus.cc:2804] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:04:57.456089 8093 raft_consensus.cc:697] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 1 LEADER]: Becoming Leader. State: Replica: 7fcd15804e074edb93a78daa879c498f, State: Running, Role: LEADER
I20251209 03:04:57.456135 8093 consensus_queue.cc:237] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:57.456635 7786 catalog_manager.cc:5654] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fcd15804e074edb93a78daa879c498f (127.2.170.193). New cstate: current_term: 1 leader_uuid: "7fcd15804e074edb93a78daa879c498f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } health_report { overall_health: HEALTHY } } }
I20251209 03:04:57.479367 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:04:57.481580 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:04:57.481776 2731 tablet_replica.cc:333] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f: stopping tablet replica
I20251209 03:04:57.481850 2731 raft_consensus.cc:2243] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:04:57.481915 2731 pending_rounds.cc:70] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f: Trying to abort 1 pending ops.
I20251209 03:04:57.481936 2731 pending_rounds.cc:77] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7230461941580808192 op_type: NO_OP noop_request { }
I20251209 03:04:57.481990 2731 raft_consensus.cc:2889] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251209 03:04:57.482020 2731 raft_consensus.cc:2272] T 02e29391c5464fd589524d831908bf32 P 7fcd15804e074edb93a78daa879c498f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:04:57.494156 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:04:58.096181 7840 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:04:58.096324 7840 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:04:58.096357 7840 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:04:59.008122 8099 raft_consensus.cc:493] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:04:59.008230 8099 raft_consensus.cc:515] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:59.008394 8099 leader_election.cc:290] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea7dc5c967df4885abae3266d546830a (127.2.170.194:39995), 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191)
W20251209 03:04:59.008776 8006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.170.193:41191: connect: Connection refused (error 111)
W20251209 03:04:59.009196 8006 leader_election.cc:336] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191): Network error: Client connection negotiation failed: client connection to 127.2.170.193:41191: connect: Connection refused (error 111)
I20251209 03:04:59.011032 7965 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02e29391c5464fd589524d831908bf32" candidate_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7dc5c967df4885abae3266d546830a" is_pre_election: true
I20251209 03:04:59.011119 7965 raft_consensus.cc:2468] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b2dd07c0754f6d8fed1907f0e09045 in term 1.
I20251209 03:04:59.011245 8005 leader_election.cc:304] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c3b2dd07c0754f6d8fed1907f0e09045, ea7dc5c967df4885abae3266d546830a; no voters: 7fcd15804e074edb93a78daa879c498f
I20251209 03:04:59.011343 8099 raft_consensus.cc:2804] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:04:59.011394 8099 raft_consensus.cc:493] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:04:59.011415 8099 raft_consensus.cc:3060] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:04:59.012120 8099 raft_consensus.cc:515] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:59.012255 8099 leader_election.cc:290] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [CANDIDATE]: Term 2 election: Requested vote from peers ea7dc5c967df4885abae3266d546830a (127.2.170.194:39995), 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191)
I20251209 03:04:59.012437 7965 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02e29391c5464fd589524d831908bf32" candidate_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7dc5c967df4885abae3266d546830a"
I20251209 03:04:59.012506 7965 raft_consensus.cc:3060] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 1 FOLLOWER]: Advancing to term 2
W20251209 03:04:59.012802 8006 leader_election.cc:336] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191): Network error: Client connection negotiation failed: client connection to 127.2.170.193:41191: connect: Connection refused (error 111)
I20251209 03:04:59.013087 7965 raft_consensus.cc:2468] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b2dd07c0754f6d8fed1907f0e09045 in term 2.
I20251209 03:04:59.013248 8005 leader_election.cc:304] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c3b2dd07c0754f6d8fed1907f0e09045, ea7dc5c967df4885abae3266d546830a; no voters: 7fcd15804e074edb93a78daa879c498f
I20251209 03:04:59.013335 8099 raft_consensus.cc:2804] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:04:59.013460 8099 raft_consensus.cc:697] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 2 LEADER]: Becoming Leader. State: Replica: c3b2dd07c0754f6d8fed1907f0e09045, State: Running, Role: LEADER
I20251209 03:04:59.013512 8099 consensus_queue.cc:237] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } }
I20251209 03:04:59.014101 7786 catalog_manager.cc:5654] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 reported cstate change: term changed from 1 to 2, leader changed from 7fcd15804e074edb93a78daa879c498f (127.2.170.193) to c3b2dd07c0754f6d8fed1907f0e09045 (127.2.170.195). New cstate: current_term: 2 leader_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3b2dd07c0754f6d8fed1907f0e09045" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fcd15804e074edb93a78daa879c498f" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 41191 } health_report { overall_health: UNKNOWN } } }
I20251209 03:04:59.435818 7965 raft_consensus.cc:1275] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 2 FOLLOWER]: Refusing update from remote peer c3b2dd07c0754f6d8fed1907f0e09045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251209 03:04:59.436107 8099 consensus_queue.cc:1048] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea7dc5c967df4885abae3266d546830a" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 39995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251209 03:04:59.437776 8006 consensus_peers.cc:597] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 -> Peer 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191): Couldn't send request to peer 7fcd15804e074edb93a78daa879c498f. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:41191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:00.096511 7840 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:05:00.096640 7840 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:05:00.096707 7840 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251209 03:05:02.004151 8006 consensus_peers.cc:597] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 -> Peer 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191): Couldn't send request to peer 7fcd15804e074edb93a78daa879c498f. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:41191: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20251209 03:05:02.096853 7840 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:05:02.096985 7840 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:05:02.097018 7840 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:05:04.097208 7840 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:05:04.097352 7840 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:05:04.097381 7840 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251209 03:05:04.163868 8006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.170.193:41191: connect: Connection refused (error 111) [suppressed 10 similar messages]
W20251209 03:05:04.547829 8006 consensus_peers.cc:597] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 -> Peer 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191): Couldn't send request to peer 7fcd15804e074edb93a78daa879c498f. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:41191: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20251209 03:05:06.097537 7840 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:05:06.097707 7840 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:05:06.097743 7840 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
W20251209 03:05:06.757331 8006 consensus_peers.cc:597] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 -> Peer 7fcd15804e074edb93a78daa879c498f (127.2.170.193:41191): Couldn't send request to peer 7fcd15804e074edb93a78daa879c498f. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:41191: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251209 03:05:08.097896 7840 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20251209 03:05:08.098035 7840 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20251209 03:05:08.098058 7840 auto_leader_rebalancer.cc:451] All tables' leader rebalancing finished this round
I20251209 03:05:08.545617 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:05:08.547915 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:08.548033 2731 tablet_replica.cc:333] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a: stopping tablet replica
I20251209 03:05:08.548082 2731 raft_consensus.cc:2243] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:08.548130 2731 raft_consensus.cc:2272] T 02e29391c5464fd589524d831908bf32 P ea7dc5c967df4885abae3266d546830a [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:08.559643 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:05:08.560453 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:05:08.562747 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:08.562851 2731 tablet_replica.cc:333] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045: stopping tablet replica
I20251209 03:05:08.562906 2731 raft_consensus.cc:2243] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:05:08.562975 2731 raft_consensus.cc:2272] T 02e29391c5464fd589524d831908bf32 P c3b2dd07c0754f6d8fed1907f0e09045 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:08.574702 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:05:08.575517 2731 master.cc:561] Master@127.2.170.254:41219 shutting down...
I20251209 03:05:08.577752 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:08.577843 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:08.577872 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P ad8eb869542e407193df6f3d312b89e5: stopping tablet replica
I20251209 03:05:08.589367 2731 master.cc:583] Master@127.2.170.254:41219 shutdown complete.
[ OK ] AutoRebalancerTest.NoReplicaMovesIfCannotMeetReplicationFactor (12513 ms)
[ RUN ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering
I20251209 03:05:08.593302 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:41003
I20251209 03:05:08.593442 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:05:08.594857 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:08.594940 8137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:08.595013 8140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:08.595151 8138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:08.595288 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:08.595326 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:08.595340 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249508595340 us; error 0 us; skew 500 ppm
I20251209 03:05:08.595875 2731 webserver.cc:492] Webserver started at http://127.2.170.254:43845/ using document root <none> and password file <none>
I20251209 03:05:08.595955 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:08.595988 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:08.596030 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:08.596266 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "88d79c456f554476aeb296b43979728d"
format_stamp: "Formatted at 2025-12-09 03:05:08 on dist-test-slave-4cb0"
I20251209 03:05:08.597141 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:08.597766 8145 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.597913 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.597956 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "88d79c456f554476aeb296b43979728d"
format_stamp: "Formatted at 2025-12-09 03:05:08 on dist-test-slave-4cb0"
I20251209 03:05:08.598003 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:08.608332 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:08.608515 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:08.610955 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:41003
I20251209 03:05:08.612681 8197 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:41003 every 8 connection(s)
I20251209 03:05:08.612829 8198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:08.613870 8198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d: Bootstrap starting.
I20251209 03:05:08.614114 8198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:08.614560 8198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d: No bootstrap required, opened a new log
I20251209 03:05:08.614666 8198 raft_consensus.cc:359] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d79c456f554476aeb296b43979728d" member_type: VOTER }
I20251209 03:05:08.614714 8198 raft_consensus.cc:385] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:08.614727 8198 raft_consensus.cc:740] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d79c456f554476aeb296b43979728d, State: Initialized, Role: FOLLOWER
I20251209 03:05:08.614761 8198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d79c456f554476aeb296b43979728d" member_type: VOTER }
I20251209 03:05:08.614799 8198 raft_consensus.cc:399] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:05:08.614815 8198 raft_consensus.cc:493] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:05:08.614840 8198 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:08.615264 8198 raft_consensus.cc:515] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d79c456f554476aeb296b43979728d" member_type: VOTER }
I20251209 03:05:08.615315 8198 leader_election.cc:304] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d79c456f554476aeb296b43979728d; no voters:
I20251209 03:05:08.615396 8198 leader_election.cc:290] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:05:08.615450 8201 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:08.615512 8198 sys_catalog.cc:565] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:05:08.615599 8201 raft_consensus.cc:697] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 1 LEADER]: Becoming Leader. State: Replica: 88d79c456f554476aeb296b43979728d, State: Running, Role: LEADER
I20251209 03:05:08.615653 8201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d79c456f554476aeb296b43979728d" member_type: VOTER }
I20251209 03:05:08.615850 8202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88d79c456f554476aeb296b43979728d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d79c456f554476aeb296b43979728d" member_type: VOTER } }
I20251209 03:05:08.615864 8203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d79c456f554476aeb296b43979728d. Latest consensus state: current_term: 1 leader_uuid: "88d79c456f554476aeb296b43979728d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d79c456f554476aeb296b43979728d" member_type: VOTER } }
I20251209 03:05:08.615962 8202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [sys.catalog]: This master's current role is: LEADER
I20251209 03:05:08.616273 8211 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:05:08.616282 8203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [sys.catalog]: This master's current role is: LEADER
I20251209 03:05:08.616417 8211 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:05:08.616783 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:05:08.617003 8211 catalog_manager.cc:1357] Generated new cluster ID: e3b8cd29f9bf440aad4c3ad0e46b886a
I20251209 03:05:08.617035 8211 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:05:08.639716 8211 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:05:08.640151 8211 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:05:08.645337 8211 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d: Generated new TSK 0
I20251209 03:05:08.645414 8211 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:05:08.648579 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:08.649940 8219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:08.650058 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:08.650003 8220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:08.650226 8222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:08.650339 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:08.650378 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:08.650395 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249508650395 us; error 0 us; skew 500 ppm
I20251209 03:05:08.651098 2731 webserver.cc:492] Webserver started at http://127.2.170.193:45133/ using document root <none> and password file <none>
I20251209 03:05:08.651197 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:08.651244 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:08.651293 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:08.651602 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "d864b254ec5d4cfd8453c7abb6a8f315"
format_stamp: "Formatted at 2025-12-09 03:05:08 on dist-test-slave-4cb0"
I20251209 03:05:08.652390 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:08.652792 8227 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.652917 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.652961 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "d864b254ec5d4cfd8453c7abb6a8f315"
format_stamp: "Formatted at 2025-12-09 03:05:08 on dist-test-slave-4cb0"
I20251209 03:05:08.653009 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:08.663550 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:08.663765 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:08.663905 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:08.664084 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:08.664110 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.664139 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:08.664163 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.666837 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:39167
I20251209 03:05:08.666883 8290 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:39167 every 8 connection(s)
I20251209 03:05:08.667183 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:08.668402 8295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:08.668434 8298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:08.668591 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:08.668500 8296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:08.668794 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:08.668838 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:08.668853 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249508668853 us; error 0 us; skew 500 ppm
I20251209 03:05:08.669503 2731 webserver.cc:492] Webserver started at http://127.2.170.194:33911/ using document root <none> and password file <none>
I20251209 03:05:08.669587 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:08.669632 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:08.669696 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:08.670017 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "adf4c1aee58f48a2abd6612a80194acb"
format_stamp: "Formatted at 2025-12-09 03:05:08 on dist-test-slave-4cb0"
I20251209 03:05:08.670897 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:08.671314 8303 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.671440 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.671480 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "adf4c1aee58f48a2abd6612a80194acb"
format_stamp: "Formatted at 2025-12-09 03:05:08 on dist-test-slave-4cb0"
I20251209 03:05:08.671523 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:08.674506 8291 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41003
I20251209 03:05:08.674571 8291 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:08.674767 8291 heartbeater.cc:507] Master 127.2.170.254:41003 requested a full tablet report, sending...
I20251209 03:05:08.675091 8162 ts_manager.cc:194] Registered new tserver with Master: d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167)
I20251209 03:05:08.675650 8162 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47316
I20251209 03:05:08.695839 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:08.696064 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:08.696224 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:08.696425 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:08.696453 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.696483 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:08.696503 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.699254 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:44687
I20251209 03:05:08.699287 8366 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:44687 every 8 connection(s)
I20251209 03:05:08.699616 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:08.700973 8370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:08.701013 8371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:08.701134 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:08.701184 8373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:08.701333 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:08.701382 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:08.701395 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249508701395 us; error 0 us; skew 500 ppm
I20251209 03:05:08.702069 2731 webserver.cc:492] Webserver started at http://127.2.170.195:46501/ using document root <none> and password file <none>
I20251209 03:05:08.702162 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:08.702214 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:08.702262 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:08.702545 8367 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41003
I20251209 03:05:08.702601 8367 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:08.702591 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "8039d77cdb19445291bd2948b041d44d"
format_stamp: "Formatted at 2025-12-09 03:05:08 on dist-test-slave-4cb0"
I20251209 03:05:08.702713 8367 heartbeater.cc:507] Master 127.2.170.254:41003 requested a full tablet report, sending...
I20251209 03:05:08.702910 8162 ts_manager.cc:194] Registered new tserver with Master: adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687)
I20251209 03:05:08.703236 8162 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47318
I20251209 03:05:08.703483 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:08.703941 8378 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.704070 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.704111 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "8039d77cdb19445291bd2948b041d44d"
format_stamp: "Formatted at 2025-12-09 03:05:08 on dist-test-slave-4cb0"
I20251209 03:05:08.704154 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:08.712602 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:08.712841 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:08.713001 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:08.713181 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:08.713208 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.713238 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:08.713263 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:08.716341 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:40809
I20251209 03:05:08.716415 8441 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:40809 every 8 connection(s)
I20251209 03:05:08.719332 8442 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41003
I20251209 03:05:08.719401 8442 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:08.719511 8442 heartbeater.cc:507] Master 127.2.170.254:41003 requested a full tablet report, sending...
I20251209 03:05:08.719800 8162 ts_manager.cc:194] Registered new tserver with Master: 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:08.719812 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003333739s
I20251209 03:05:08.720170 8162 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47324
I20251209 03:05:09.676136 8291 heartbeater.cc:499] Master 127.2.170.254:41003 was elected leader, sending a full tablet report...
I20251209 03:05:09.703733 8367 heartbeater.cc:499] Master 127.2.170.254:41003 was elected leader, sending a full tablet report...
I20251209 03:05:09.720647 8442 heartbeater.cc:499] Master 127.2.170.254:41003 was elected leader, sending a full tablet report...
I20251209 03:05:09.744709 2731 test_util.cc:276] Using random seed: 771219521
I20251209 03:05:09.749580 8162 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45030:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:05:09.749790 8162 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:05:09.754549 8331 tablet_service.cc:1505] Processing CreateTablet for tablet 4eed7112e439482fab13e05b7998f41b (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:09.754611 8330 tablet_service.cc:1505] Processing CreateTablet for tablet 54cc03f6fe5a4bc9899442e38a6d95bb (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:09.754873 8329 tablet_service.cc:1505] Processing CreateTablet for tablet c1cbd31efb904dfd83df937ad0fdfe5d (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:09.754992 8328 tablet_service.cc:1505] Processing CreateTablet for tablet 97ea2cd32b4742ce8fb65ca0b26be071 (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:09.755213 8406 tablet_service.cc:1505] Processing CreateTablet for tablet 4eed7112e439482fab13e05b7998f41b (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:09.755262 8405 tablet_service.cc:1505] Processing CreateTablet for tablet 54cc03f6fe5a4bc9899442e38a6d95bb (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:09.755319 8406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eed7112e439482fab13e05b7998f41b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.755383 8404 tablet_service.cc:1505] Processing CreateTablet for tablet c1cbd31efb904dfd83df937ad0fdfe5d (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:09.754710 8331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eed7112e439482fab13e05b7998f41b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.755462 8404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1cbd31efb904dfd83df937ad0fdfe5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.755496 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 54cc03f6fe5a4bc9899442e38a6d95bb (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:09.755602 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54cc03f6fe5a4bc9899442e38a6d95bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.755839 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 97ea2cd32b4742ce8fb65ca0b26be071 (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:09.755899 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ea2cd32b4742ce8fb65ca0b26be071. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.756847 8403 tablet_service.cc:1505] Processing CreateTablet for tablet 97ea2cd32b4742ce8fb65ca0b26be071 (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:09.756943 8403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ea2cd32b4742ce8fb65ca0b26be071. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.756996 8463 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb: Bootstrap starting.
I20251209 03:05:09.757098 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 4eed7112e439482fab13e05b7998f41b (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:09.756958 8328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ea2cd32b4742ce8fb65ca0b26be071. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.757210 8462 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d: Bootstrap starting.
I20251209 03:05:09.757244 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eed7112e439482fab13e05b7998f41b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.755769 8253 tablet_service.cc:1505] Processing CreateTablet for tablet c1cbd31efb904dfd83df937ad0fdfe5d (DEFAULT_TABLE table=test-workload [id=20be529bee674a97bca640ab8addd641]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:09.758045 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1cbd31efb904dfd83df937ad0fdfe5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.758152 8405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54cc03f6fe5a4bc9899442e38a6d95bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.757265 8330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54cc03f6fe5a4bc9899442e38a6d95bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.759819 8463 tablet_bootstrap.cc:654] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.760331 8463 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb: No bootstrap required, opened a new log
I20251209 03:05:09.760360 8463 ts_tablet_manager.cc:1403] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251209 03:05:09.760389 8329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1cbd31efb904dfd83df937ad0fdfe5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:09.760514 8463 raft_consensus.cc:359] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.760583 8463 raft_consensus.cc:385] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.760597 8463 raft_consensus.cc:740] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf4c1aee58f48a2abd6612a80194acb, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.760634 8463 consensus_queue.cc:260] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.760763 8463 ts_tablet_manager.cc:1434] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:09.760818 8463 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb: Bootstrap starting.
I20251209 03:05:09.761188 8463 tablet_bootstrap.cc:654] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.761729 8463 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb: No bootstrap required, opened a new log
I20251209 03:05:09.761785 8463 ts_tablet_manager.cc:1403] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:09.761946 8463 raft_consensus.cc:359] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.762061 8463 raft_consensus.cc:385] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.762120 8463 raft_consensus.cc:740] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf4c1aee58f48a2abd6612a80194acb, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.762203 8463 consensus_queue.cc:260] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.762341 8463 ts_tablet_manager.cc:1434] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:09.762423 8463 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb: Bootstrap starting.
I20251209 03:05:09.762820 8463 tablet_bootstrap.cc:654] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.763125 8462 tablet_bootstrap.cc:654] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.763355 8463 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb: No bootstrap required, opened a new log
I20251209 03:05:09.763396 8463 ts_tablet_manager.cc:1403] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:09.763456 8464 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315: Bootstrap starting.
I20251209 03:05:09.763556 8463 raft_consensus.cc:359] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.763612 8463 raft_consensus.cc:385] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.763630 8463 raft_consensus.cc:740] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf4c1aee58f48a2abd6612a80194acb, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.763676 8463 consensus_queue.cc:260] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.763756 8463 ts_tablet_manager.cc:1434] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:09.763818 8463 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb: Bootstrap starting.
I20251209 03:05:09.763854 8464 tablet_bootstrap.cc:654] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.764120 8463 tablet_bootstrap.cc:654] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.764284 8462 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d: No bootstrap required, opened a new log
I20251209 03:05:09.764339 8462 ts_tablet_manager.cc:1403] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20251209 03:05:09.764482 8462 raft_consensus.cc:359] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.764544 8462 raft_consensus.cc:385] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.764564 8462 raft_consensus.cc:740] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8039d77cdb19445291bd2948b041d44d, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.764631 8462 consensus_queue.cc:260] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.764690 8463 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb: No bootstrap required, opened a new log
I20251209 03:05:09.764720 8463 ts_tablet_manager.cc:1403] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:09.764804 8462 ts_tablet_manager.cc:1434] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:09.764859 8463 raft_consensus.cc:359] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.764926 8463 raft_consensus.cc:385] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.764974 8463 raft_consensus.cc:740] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf4c1aee58f48a2abd6612a80194acb, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.765005 8464 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315: No bootstrap required, opened a new log
I20251209 03:05:09.765025 8463 consensus_queue.cc:260] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.765120 8464 ts_tablet_manager.cc:1403] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:09.765163 8463 ts_tablet_manager.cc:1434] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:09.764866 8462 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d: Bootstrap starting.
I20251209 03:05:09.765324 8464 raft_consensus.cc:359] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.765452 8464 raft_consensus.cc:385] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.765580 8464 raft_consensus.cc:740] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d864b254ec5d4cfd8453c7abb6a8f315, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.765734 8464 consensus_queue.cc:260] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.765834 8462 tablet_bootstrap.cc:654] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.765981 8464 ts_tablet_manager.cc:1434] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:09.766084 8464 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315: Bootstrap starting.
I20251209 03:05:09.766531 8464 tablet_bootstrap.cc:654] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.766606 8462 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d: No bootstrap required, opened a new log
I20251209 03:05:09.766652 8462 ts_tablet_manager.cc:1403] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:09.766783 8462 raft_consensus.cc:359] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.766829 8462 raft_consensus.cc:385] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.766841 8462 raft_consensus.cc:740] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8039d77cdb19445291bd2948b041d44d, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.766881 8462 consensus_queue.cc:260] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.766978 8462 ts_tablet_manager.cc:1434] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:09.767040 8462 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d: Bootstrap starting.
I20251209 03:05:09.767127 8464 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315: No bootstrap required, opened a new log
I20251209 03:05:09.767179 8464 ts_tablet_manager.cc:1403] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:09.767335 8464 raft_consensus.cc:359] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.767408 8464 raft_consensus.cc:385] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.767418 8462 tablet_bootstrap.cc:654] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.767472 8464 raft_consensus.cc:740] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d864b254ec5d4cfd8453c7abb6a8f315, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.767565 8464 consensus_queue.cc:260] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.767688 8464 ts_tablet_manager.cc:1434] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:09.767774 8464 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315: Bootstrap starting.
I20251209 03:05:09.768191 8464 tablet_bootstrap.cc:654] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.768739 8464 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315: No bootstrap required, opened a new log
I20251209 03:05:09.768779 8464 ts_tablet_manager.cc:1403] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:09.768877 8464 raft_consensus.cc:359] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.768916 8464 raft_consensus.cc:385] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.768927 8464 raft_consensus.cc:740] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d864b254ec5d4cfd8453c7abb6a8f315, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.768954 8464 consensus_queue.cc:260] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.769025 8464 ts_tablet_manager.cc:1434] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:09.769114 8464 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315: Bootstrap starting.
I20251209 03:05:09.769173 8462 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d: No bootstrap required, opened a new log
I20251209 03:05:09.769217 8462 ts_tablet_manager.cc:1403] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:09.769465 8464 tablet_bootstrap.cc:654] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.769436 8462 raft_consensus.cc:359] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.769579 8462 raft_consensus.cc:385] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.769613 8462 raft_consensus.cc:740] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8039d77cdb19445291bd2948b041d44d, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.769655 8462 consensus_queue.cc:260] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.770130 8464 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315: No bootstrap required, opened a new log
I20251209 03:05:09.770188 8464 ts_tablet_manager.cc:1403] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:09.770136 8462 ts_tablet_manager.cc:1434] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:09.770318 8464 raft_consensus.cc:359] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.770355 8462 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d: Bootstrap starting.
I20251209 03:05:09.770876 8462 tablet_bootstrap.cc:654] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:09.770370 8464 raft_consensus.cc:385] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.771169 8464 raft_consensus.cc:740] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d864b254ec5d4cfd8453c7abb6a8f315, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.771205 8464 consensus_queue.cc:260] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.771315 8464 ts_tablet_manager.cc:1434] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:09.771478 8462 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d: No bootstrap required, opened a new log
I20251209 03:05:09.771528 8462 ts_tablet_manager.cc:1403] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:09.771687 8462 raft_consensus.cc:359] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.771751 8462 raft_consensus.cc:385] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:09.771770 8462 raft_consensus.cc:740] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8039d77cdb19445291bd2948b041d44d, State: Initialized, Role: FOLLOWER
I20251209 03:05:09.771842 8462 consensus_queue.cc:260] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.771927 8462 ts_tablet_manager.cc:1434] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:09.777035 8472 raft_consensus.cc:493] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:09.777096 8472 raft_consensus.cc:515] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.777258 8472 leader_election.cc:290] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167), adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687)
I20251209 03:05:09.779666 8265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "97ea2cd32b4742ce8fb65ca0b26be071" candidate_uuid: "8039d77cdb19445291bd2948b041d44d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" is_pre_election: true
I20251209 03:05:09.779703 8341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "97ea2cd32b4742ce8fb65ca0b26be071" candidate_uuid: "8039d77cdb19445291bd2948b041d44d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf4c1aee58f48a2abd6612a80194acb" is_pre_election: true
I20251209 03:05:09.779745 8265 raft_consensus.cc:2468] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8039d77cdb19445291bd2948b041d44d in term 0.
I20251209 03:05:09.779762 8341 raft_consensus.cc:2468] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8039d77cdb19445291bd2948b041d44d in term 0.
I20251209 03:05:09.779883 8379 leader_election.cc:304] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8039d77cdb19445291bd2948b041d44d, adf4c1aee58f48a2abd6612a80194acb; no voters:
I20251209 03:05:09.779969 8472 raft_consensus.cc:2804] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:09.780015 8472 raft_consensus.cc:493] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:09.780035 8472 raft_consensus.cc:3060] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.780555 8472 raft_consensus.cc:515] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.780658 8472 leader_election.cc:290] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [CANDIDATE]: Term 1 election: Requested vote from peers d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167), adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687)
I20251209 03:05:09.780788 8341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "97ea2cd32b4742ce8fb65ca0b26be071" candidate_uuid: "8039d77cdb19445291bd2948b041d44d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf4c1aee58f48a2abd6612a80194acb"
I20251209 03:05:09.780791 8265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "97ea2cd32b4742ce8fb65ca0b26be071" candidate_uuid: "8039d77cdb19445291bd2948b041d44d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d864b254ec5d4cfd8453c7abb6a8f315"
I20251209 03:05:09.780848 8341 raft_consensus.cc:3060] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.780864 8265 raft_consensus.cc:3060] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.781492 8341 raft_consensus.cc:2468] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8039d77cdb19445291bd2948b041d44d in term 1.
I20251209 03:05:09.781597 8265 raft_consensus.cc:2468] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8039d77cdb19445291bd2948b041d44d in term 1.
I20251209 03:05:09.781620 8379 leader_election.cc:304] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8039d77cdb19445291bd2948b041d44d, adf4c1aee58f48a2abd6612a80194acb; no voters:
I20251209 03:05:09.781795 8472 raft_consensus.cc:2804] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:09.781903 8472 raft_consensus.cc:697] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Becoming Leader. State: Replica: 8039d77cdb19445291bd2948b041d44d, State: Running, Role: LEADER
I20251209 03:05:09.782024 8472 consensus_queue.cc:237] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.782533 8161 catalog_manager.cc:5654] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8039d77cdb19445291bd2948b041d44d (127.2.170.195). New cstate: current_term: 1 leader_uuid: "8039d77cdb19445291bd2948b041d44d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: HEALTHY } } }
I20251209 03:05:09.849993 8466 raft_consensus.cc:493] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:09.850097 8466 raft_consensus.cc:515] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.850289 8466 leader_election.cc:290] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167), 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:09.852912 8265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "54cc03f6fe5a4bc9899442e38a6d95bb" candidate_uuid: "adf4c1aee58f48a2abd6612a80194acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" is_pre_election: true
I20251209 03:05:09.852928 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "54cc03f6fe5a4bc9899442e38a6d95bb" candidate_uuid: "adf4c1aee58f48a2abd6612a80194acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8039d77cdb19445291bd2948b041d44d" is_pre_election: true
I20251209 03:05:09.853044 8265 raft_consensus.cc:2468] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf4c1aee58f48a2abd6612a80194acb in term 0.
I20251209 03:05:09.853047 8416 raft_consensus.cc:2468] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf4c1aee58f48a2abd6612a80194acb in term 0.
I20251209 03:05:09.853209 8306 leader_election.cc:304] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8039d77cdb19445291bd2948b041d44d, adf4c1aee58f48a2abd6612a80194acb; no voters:
I20251209 03:05:09.853339 8466 raft_consensus.cc:2804] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:09.853415 8466 raft_consensus.cc:493] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:09.853440 8466 raft_consensus.cc:3060] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.854203 8466 raft_consensus.cc:515] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.854342 8466 leader_election.cc:290] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 1 election: Requested vote from peers d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167), 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:09.854503 8265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "54cc03f6fe5a4bc9899442e38a6d95bb" candidate_uuid: "adf4c1aee58f48a2abd6612a80194acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d864b254ec5d4cfd8453c7abb6a8f315"
I20251209 03:05:09.854529 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "54cc03f6fe5a4bc9899442e38a6d95bb" candidate_uuid: "adf4c1aee58f48a2abd6612a80194acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8039d77cdb19445291bd2948b041d44d"
I20251209 03:05:09.854569 8265 raft_consensus.cc:3060] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.854584 8416 raft_consensus.cc:3060] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.855152 8265 raft_consensus.cc:2468] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf4c1aee58f48a2abd6612a80194acb in term 1.
I20251209 03:05:09.855175 8416 raft_consensus.cc:2468] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf4c1aee58f48a2abd6612a80194acb in term 1.
I20251209 03:05:09.855316 8306 leader_election.cc:304] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adf4c1aee58f48a2abd6612a80194acb, d864b254ec5d4cfd8453c7abb6a8f315; no voters:
I20251209 03:05:09.855409 8466 raft_consensus.cc:2804] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:09.855621 8466 raft_consensus.cc:697] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 1 LEADER]: Becoming Leader. State: Replica: adf4c1aee58f48a2abd6612a80194acb, State: Running, Role: LEADER
I20251209 03:05:09.855683 8466 consensus_queue.cc:237] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.856292 8161 catalog_manager.cc:5654] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb reported cstate change: term changed from 0 to 1, leader changed from <none> to adf4c1aee58f48a2abd6612a80194acb (127.2.170.194). New cstate: current_term: 1 leader_uuid: "adf4c1aee58f48a2abd6612a80194acb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:09.901281 8472 raft_consensus.cc:493] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:09.901381 8472 raft_consensus.cc:515] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.901525 8472 leader_election.cc:290] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687), d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167)
I20251209 03:05:09.901757 8265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4eed7112e439482fab13e05b7998f41b" candidate_uuid: "8039d77cdb19445291bd2948b041d44d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" is_pre_election: true
I20251209 03:05:09.901880 8265 raft_consensus.cc:2468] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8039d77cdb19445291bd2948b041d44d in term 0.
I20251209 03:05:09.901764 8341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4eed7112e439482fab13e05b7998f41b" candidate_uuid: "8039d77cdb19445291bd2948b041d44d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf4c1aee58f48a2abd6612a80194acb" is_pre_election: true
I20251209 03:05:09.901991 8341 raft_consensus.cc:2468] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8039d77cdb19445291bd2948b041d44d in term 0.
I20251209 03:05:09.902042 8381 leader_election.cc:304] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8039d77cdb19445291bd2948b041d44d, d864b254ec5d4cfd8453c7abb6a8f315; no voters:
I20251209 03:05:09.902125 8472 raft_consensus.cc:2804] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:09.902165 8472 raft_consensus.cc:493] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:09.902192 8472 raft_consensus.cc:3060] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.902796 8472 raft_consensus.cc:515] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.902916 8472 leader_election.cc:290] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [CANDIDATE]: Term 1 election: Requested vote from peers adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687), d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167)
I20251209 03:05:09.903059 8265 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4eed7112e439482fab13e05b7998f41b" candidate_uuid: "8039d77cdb19445291bd2948b041d44d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d864b254ec5d4cfd8453c7abb6a8f315"
I20251209 03:05:09.903119 8265 raft_consensus.cc:3060] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.903101 8341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4eed7112e439482fab13e05b7998f41b" candidate_uuid: "8039d77cdb19445291bd2948b041d44d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf4c1aee58f48a2abd6612a80194acb"
I20251209 03:05:09.903175 8341 raft_consensus.cc:3060] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.903692 8265 raft_consensus.cc:2468] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8039d77cdb19445291bd2948b041d44d in term 1.
I20251209 03:05:09.903816 8341 raft_consensus.cc:2468] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8039d77cdb19445291bd2948b041d44d in term 1.
I20251209 03:05:09.903869 8381 leader_election.cc:304] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8039d77cdb19445291bd2948b041d44d, d864b254ec5d4cfd8453c7abb6a8f315; no voters:
I20251209 03:05:09.903990 8472 raft_consensus.cc:2804] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:09.904039 8472 raft_consensus.cc:697] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Becoming Leader. State: Replica: 8039d77cdb19445291bd2948b041d44d, State: Running, Role: LEADER
I20251209 03:05:09.904088 8472 consensus_queue.cc:237] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.904707 8161 catalog_manager.cc:5654] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8039d77cdb19445291bd2948b041d44d (127.2.170.195). New cstate: current_term: 1 leader_uuid: "8039d77cdb19445291bd2948b041d44d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: HEALTHY } } }
I20251209 03:05:09.941402 8473 raft_consensus.cc:493] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:09.941532 8473 raft_consensus.cc:515] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.941869 8473 leader_election.cc:290] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687), 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:09.945284 8341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1cbd31efb904dfd83df937ad0fdfe5d" candidate_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf4c1aee58f48a2abd6612a80194acb" is_pre_election: true
I20251209 03:05:09.945307 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1cbd31efb904dfd83df937ad0fdfe5d" candidate_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8039d77cdb19445291bd2948b041d44d" is_pre_election: true
I20251209 03:05:09.945381 8341 raft_consensus.cc:2468] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d864b254ec5d4cfd8453c7abb6a8f315 in term 0.
I20251209 03:05:09.945391 8416 raft_consensus.cc:2468] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d864b254ec5d4cfd8453c7abb6a8f315 in term 0.
I20251209 03:05:09.945528 8228 leader_election.cc:304] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adf4c1aee58f48a2abd6612a80194acb, d864b254ec5d4cfd8453c7abb6a8f315; no voters:
I20251209 03:05:09.945621 8473 raft_consensus.cc:2804] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:09.945654 8473 raft_consensus.cc:493] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:09.945691 8473 raft_consensus.cc:3060] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.946288 8473 raft_consensus.cc:515] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.946400 8473 leader_election.cc:290] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [CANDIDATE]: Term 1 election: Requested vote from peers adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687), 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:09.946545 8341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1cbd31efb904dfd83df937ad0fdfe5d" candidate_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf4c1aee58f48a2abd6612a80194acb"
I20251209 03:05:09.946578 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1cbd31efb904dfd83df937ad0fdfe5d" candidate_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8039d77cdb19445291bd2948b041d44d"
I20251209 03:05:09.946609 8341 raft_consensus.cc:3060] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.946645 8416 raft_consensus.cc:3060] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:09.947142 8341 raft_consensus.cc:2468] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d864b254ec5d4cfd8453c7abb6a8f315 in term 1.
I20251209 03:05:09.947196 8416 raft_consensus.cc:2468] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d864b254ec5d4cfd8453c7abb6a8f315 in term 1.
I20251209 03:05:09.947294 8228 leader_election.cc:304] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adf4c1aee58f48a2abd6612a80194acb, d864b254ec5d4cfd8453c7abb6a8f315; no voters:
I20251209 03:05:09.947371 8473 raft_consensus.cc:2804] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:09.947502 8473 raft_consensus.cc:697] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 LEADER]: Becoming Leader. State: Replica: d864b254ec5d4cfd8453c7abb6a8f315, State: Running, Role: LEADER
I20251209 03:05:09.947556 8473 consensus_queue.cc:237] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:09.948087 8161 catalog_manager.cc:5654] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 reported cstate change: term changed from 0 to 1, leader changed from <none> to d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:09.978477 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:05:09.980980 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:09.981142 2731 tablet_replica.cc:333] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315: stopping tablet replica
I20251209 03:05:09.981215 2731 raft_consensus.cc:2243] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:09.981269 2731 raft_consensus.cc:2272] T 4eed7112e439482fab13e05b7998f41b P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:09.981501 2731 tablet_replica.cc:333] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315: stopping tablet replica
I20251209 03:05:09.981544 2731 raft_consensus.cc:2243] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:09.981596 2731 pending_rounds.cc:70] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315: Trying to abort 1 pending ops.
I20251209 03:05:09.981622 2731 pending_rounds.cc:77] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7230461992745734144 op_type: NO_OP noop_request { }
I20251209 03:05:09.981712 2731 raft_consensus.cc:2889] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251209 03:05:09.981748 2731 raft_consensus.cc:2272] T c1cbd31efb904dfd83df937ad0fdfe5d P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:09.982125 2731 tablet_replica.cc:333] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315: stopping tablet replica
I20251209 03:05:09.982178 2731 raft_consensus.cc:2243] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:09.982215 2731 raft_consensus.cc:2272] T 97ea2cd32b4742ce8fb65ca0b26be071 P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:09.982365 2731 tablet_replica.cc:333] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315: stopping tablet replica
I20251209 03:05:09.982403 2731 raft_consensus.cc:2243] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:09.982431 2731 raft_consensus.cc:2272] T 54cc03f6fe5a4bc9899442e38a6d95bb P d864b254ec5d4cfd8453c7abb6a8f315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:09.994366 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:05:10.212486 8472 consensus_queue.cc:1048] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251209 03:05:10.213864 8381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20251209 03:05:10.214319 8381 consensus_peers.cc:597] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:10.337704 8466 consensus_queue.cc:1048] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251209 03:05:10.339401 8306 consensus_peers.cc:597] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251209 03:05:10.407111 8381 consensus_peers.cc:597] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:10.518011 8472 consensus_queue.cc:1048] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:10.994657 8472 consensus_queue.cc:579] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Leader has been unable to successfully communicate with peer d864b254ec5d4cfd8453c7abb6a8f315 for more than 1 seconds (1.090s)
W20251209 03:05:10.995337 8161 catalog_manager.cc:5332] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4eed7112e439482fab13e05b7998f41b with cas_config_opid_index -1: no extra replica candidate found for tablet 4eed7112e439482fab13e05b7998f41b (table test-workload [id=20be529bee674a97bca640ab8addd641]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (3) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
I20251209 03:05:10.995674 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:10.996904 8503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:10.997009 8505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:10.997121 8502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:10.997195 2731 server_base.cc:1047] running on GCE node
I20251209 03:05:10.997319 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:10.997351 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:10.997363 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249510997364 us; error 0 us; skew 500 ppm
I20251209 03:05:10.998066 2731 webserver.cc:492] Webserver started at http://127.2.170.196:33145/ using document root <none> and password file <none>
I20251209 03:05:10.998138 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:10.998183 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:10.998229 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:10.998556 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-3-root/instance:
uuid: "9144c65475bc402188177bb8a3fd3aa3"
format_stamp: "Formatted at 2025-12-09 03:05:10 on dist-test-slave-4cb0"
I20251209 03:05:10.999388 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:10.999815 8510 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:10.999938 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:10.999979 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-3-root
uuid: "9144c65475bc402188177bb8a3fd3aa3"
format_stamp: "Formatted at 2025-12-09 03:05:10 on dist-test-slave-4cb0"
I20251209 03:05:11.000022 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.NoRebalancingIfReplicasRecovering.1765249356537226-2731-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:11.005487 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:11.005666 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:11.005828 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:11.006011 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:11.006043 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:11.006076 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:11.006100 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:11.009662 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.196:33957
I20251209 03:05:11.010334 8573 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.196:33957 every 8 connection(s)
I20251209 03:05:11.013113 8574 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41003
I20251209 03:05:11.013183 8574 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:11.013268 8574 heartbeater.cc:507] Master 127.2.170.254:41003 requested a full tablet report, sending...
I20251209 03:05:11.013473 8161 ts_manager.cc:194] Registered new tserver with Master: 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196:33957)
I20251209 03:05:11.013880 8161 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45038
I20251209 03:05:11.017097 8416 consensus_queue.cc:237] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:11.017714 8341 raft_consensus.cc:1275] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:05:11.017971 8482 consensus_queue.cc:1048] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20251209 03:05:11.018075 8381 consensus_peers.cc:597] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:11.018503 8472 raft_consensus.cc:2955] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
I20251209 03:05:11.018610 8341 raft_consensus.cc:2955] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
I20251209 03:05:11.019388 8148 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4eed7112e439482fab13e05b7998f41b with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 2)
I20251209 03:05:11.019742 8161 catalog_manager.cc:5654] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d reported cstate change: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "8039d77cdb19445291bd2948b041d44d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251209 03:05:11.021091 8379 consensus_peers.cc:597] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d -> Peer 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196:33957): Couldn't send request to peer 9144c65475bc402188177bb8a3fd3aa3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eed7112e439482fab13e05b7998f41b. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:11.299605 8501 consensus_queue.cc:579] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Leader has been unable to successfully communicate with peer d864b254ec5d4cfd8453c7abb6a8f315 for more than 1 seconds (1.517s)
I20251209 03:05:11.300463 8416 consensus_queue.cc:237] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
W20251209 03:05:11.301091 8379 consensus_peers.cc:597] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d -> Peer 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196:33957): Couldn't send request to peer 9144c65475bc402188177bb8a3fd3aa3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97ea2cd32b4742ce8fb65ca0b26be071. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:11.301098 8341 raft_consensus.cc:1275] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251209 03:05:11.301242 8381 consensus_peers.cc:597] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:11.301277 8482 consensus_queue.cc:1048] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:11.302044 8501 raft_consensus.cc:2955] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
I20251209 03:05:11.302203 8341 raft_consensus.cc:2955] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
I20251209 03:05:11.303061 8148 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 97ea2cd32b4742ce8fb65ca0b26be071 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251209 03:05:11.303169 8162 catalog_manager.cc:5654] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d reported cstate change: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "8039d77cdb19445291bd2948b041d44d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:11.401250 8498 consensus_queue.cc:579] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Leader has been unable to successfully communicate with peer d864b254ec5d4cfd8453c7abb6a8f315 for more than 1 seconds (1.545s)
I20251209 03:05:11.402160 8341 consensus_queue.cc:237] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:11.402781 8416 raft_consensus.cc:1275] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251209 03:05:11.402966 8306 consensus_peers.cc:597] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:11.402974 8588 consensus_queue.cc:1048] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:11.403690 8587 raft_consensus.cc:2955] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
I20251209 03:05:11.403800 8416 raft_consensus.cc:2955] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
I20251209 03:05:11.404906 8146 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 54cc03f6fe5a4bc9899442e38a6d95bb with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251209 03:05:11.404940 8161 catalog_manager.cc:5654] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d reported cstate change: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "adf4c1aee58f48a2abd6612a80194acb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
W20251209 03:05:11.405814 8304 consensus_peers.cc:597] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb -> Peer 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196:33957): Couldn't send request to peer 9144c65475bc402188177bb8a3fd3aa3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 54cc03f6fe5a4bc9899442e38a6d95bb. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:11.579030 8595 ts_tablet_manager.cc:933] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: Initiating tablet copy from peer 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:11.579246 8595 tablet_copy_client.cc:323] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.195:40809
I20251209 03:05:11.581887 8426 tablet_copy_service.cc:140] P 8039d77cdb19445291bd2948b041d44d: Received BeginTabletCopySession request for tablet 4eed7112e439482fab13e05b7998f41b from peer 9144c65475bc402188177bb8a3fd3aa3 ({username='slave'} at 127.0.0.1:34584)
I20251209 03:05:11.581949 8426 tablet_copy_service.cc:161] P 8039d77cdb19445291bd2948b041d44d: Beginning new tablet copy session on tablet 4eed7112e439482fab13e05b7998f41b from peer 9144c65475bc402188177bb8a3fd3aa3 at {username='slave'} at 127.0.0.1:34584: session id = 9144c65475bc402188177bb8a3fd3aa3-4eed7112e439482fab13e05b7998f41b
I20251209 03:05:11.582379 8426 tablet_copy_source_session.cc:215] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:05:11.582810 8595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eed7112e439482fab13e05b7998f41b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:11.583907 8595 tablet_copy_client.cc:806] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Starting download of 0 data blocks...
I20251209 03:05:11.584018 8595 tablet_copy_client.cc:670] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Starting download of 1 WAL segments...
I20251209 03:05:11.584553 8595 tablet_copy_client.cc:538] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:05:11.585100 8595 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap starting.
I20251209 03:05:11.586133 8595 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:11.586277 8595 tablet_bootstrap.cc:492] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap complete.
I20251209 03:05:11.586364 8595 ts_tablet_manager.cc:1403] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:11.586483 8595 raft_consensus.cc:359] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:11.586565 8595 raft_consensus.cc:740] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9144c65475bc402188177bb8a3fd3aa3, State: Initialized, Role: LEARNER
I20251209 03:05:11.586589 8595 consensus_queue.cc:260] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:11.586745 8595 ts_tablet_manager.cc:1434] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251209 03:05:11.586800 8574 heartbeater.cc:499] Master 127.2.170.254:41003 was elected leader, sending a full tablet report...
I20251209 03:05:11.586930 8426 tablet_copy_service.cc:342] P 8039d77cdb19445291bd2948b041d44d: Request end of tablet copy session 9144c65475bc402188177bb8a3fd3aa3-4eed7112e439482fab13e05b7998f41b received from {username='slave'} at 127.0.0.1:34584
I20251209 03:05:11.586987 8426 tablet_copy_service.cc:434] P 8039d77cdb19445291bd2948b041d44d: ending tablet copy session 9144c65475bc402188177bb8a3fd3aa3-4eed7112e439482fab13e05b7998f41b on tablet 4eed7112e439482fab13e05b7998f41b with peer 9144c65475bc402188177bb8a3fd3aa3
W20251209 03:05:11.616914 8210 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver d864b254ec5d4cfd8453c7abb6a8f315 not available for placement
I20251209 03:05:11.745630 8595 ts_tablet_manager.cc:933] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: Initiating tablet copy from peer 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:11.745884 8595 tablet_copy_client.cc:323] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.195:40809
I20251209 03:05:11.746073 8426 tablet_copy_service.cc:140] P 8039d77cdb19445291bd2948b041d44d: Received BeginTabletCopySession request for tablet 97ea2cd32b4742ce8fb65ca0b26be071 from peer 9144c65475bc402188177bb8a3fd3aa3 ({username='slave'} at 127.0.0.1:34584)
I20251209 03:05:11.746117 8426 tablet_copy_service.cc:161] P 8039d77cdb19445291bd2948b041d44d: Beginning new tablet copy session on tablet 97ea2cd32b4742ce8fb65ca0b26be071 from peer 9144c65475bc402188177bb8a3fd3aa3 at {username='slave'} at 127.0.0.1:34584: session id = 9144c65475bc402188177bb8a3fd3aa3-97ea2cd32b4742ce8fb65ca0b26be071
I20251209 03:05:11.746688 8426 tablet_copy_source_session.cc:215] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:05:11.747042 8595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ea2cd32b4742ce8fb65ca0b26be071. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:11.748960 8595 tablet_copy_client.cc:806] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Starting download of 0 data blocks...
I20251209 03:05:11.749054 8595 tablet_copy_client.cc:670] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Starting download of 1 WAL segments...
I20251209 03:05:11.749666 8595 tablet_copy_client.cc:538] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:05:11.750248 8595 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap starting.
I20251209 03:05:11.751225 8595 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:11.751350 8595 tablet_bootstrap.cc:492] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap complete.
I20251209 03:05:11.751413 8595 ts_tablet_manager.cc:1403] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:11.751547 8595 raft_consensus.cc:359] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:11.751607 8595 raft_consensus.cc:740] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9144c65475bc402188177bb8a3fd3aa3, State: Initialized, Role: LEARNER
I20251209 03:05:11.751627 8595 consensus_queue.cc:260] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:11.751695 8595 ts_tablet_manager.cc:1434] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:11.751920 8426 tablet_copy_service.cc:342] P 8039d77cdb19445291bd2948b041d44d: Request end of tablet copy session 9144c65475bc402188177bb8a3fd3aa3-97ea2cd32b4742ce8fb65ca0b26be071 received from {username='slave'} at 127.0.0.1:34584
I20251209 03:05:11.751955 8588 raft_consensus.cc:493] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:11.751988 8426 tablet_copy_service.cc:434] P 8039d77cdb19445291bd2948b041d44d: ending tablet copy session 9144c65475bc402188177bb8a3fd3aa3-97ea2cd32b4742ce8fb65ca0b26be071 on tablet 97ea2cd32b4742ce8fb65ca0b26be071 with peer 9144c65475bc402188177bb8a3fd3aa3
I20251209 03:05:11.752000 8588 raft_consensus.cc:515] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:11.752107 8588 leader_election.cc:290] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167), 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:11.752370 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1cbd31efb904dfd83df937ad0fdfe5d" candidate_uuid: "adf4c1aee58f48a2abd6612a80194acb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8039d77cdb19445291bd2948b041d44d" is_pre_election: true
I20251209 03:05:11.752444 8416 raft_consensus.cc:2468] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf4c1aee58f48a2abd6612a80194acb in term 1.
I20251209 03:05:11.752552 8306 leader_election.cc:304] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8039d77cdb19445291bd2948b041d44d, adf4c1aee58f48a2abd6612a80194acb; no voters:
I20251209 03:05:11.752635 8588 raft_consensus.cc:2804] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:05:11.752677 8588 raft_consensus.cc:493] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251209 03:05:11.752676 8306 leader_election.cc:336] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
I20251209 03:05:11.752710 8588 raft_consensus.cc:3060] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:11.753314 8588 raft_consensus.cc:515] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:11.753396 8588 leader_election.cc:290] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 2 election: Requested vote from peers d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167), 8039d77cdb19445291bd2948b041d44d (127.2.170.195:40809)
I20251209 03:05:11.753654 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1cbd31efb904dfd83df937ad0fdfe5d" candidate_uuid: "adf4c1aee58f48a2abd6612a80194acb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8039d77cdb19445291bd2948b041d44d"
I20251209 03:05:11.753743 8416 raft_consensus.cc:3060] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Advancing to term 2
W20251209 03:05:11.753851 8306 leader_election.cc:336] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
I20251209 03:05:11.754257 8416 raft_consensus.cc:2468] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf4c1aee58f48a2abd6612a80194acb in term 2.
I20251209 03:05:11.754413 8306 leader_election.cc:304] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8039d77cdb19445291bd2948b041d44d, adf4c1aee58f48a2abd6612a80194acb; no voters: d864b254ec5d4cfd8453c7abb6a8f315
I20251209 03:05:11.754498 8588 raft_consensus.cc:2804] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:05:11.754568 8588 raft_consensus.cc:697] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 2 LEADER]: Becoming Leader. State: Replica: adf4c1aee58f48a2abd6612a80194acb, State: Running, Role: LEADER
I20251209 03:05:11.754616 8588 consensus_queue.cc:237] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } }
I20251209 03:05:11.755064 8162 catalog_manager.cc:5654] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb reported cstate change: term changed from 1 to 2, leader changed from d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) to adf4c1aee58f48a2abd6612a80194acb (127.2.170.194). New cstate: current_term: 2 leader_uuid: "adf4c1aee58f48a2abd6612a80194acb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:11.956359 8548 raft_consensus.cc:1217] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251209 03:05:11.979755 8595 ts_tablet_manager.cc:933] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: Initiating tablet copy from peer adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687)
I20251209 03:05:11.979933 8595 tablet_copy_client.cc:323] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:44687
I20251209 03:05:11.982626 8351 tablet_copy_service.cc:140] P adf4c1aee58f48a2abd6612a80194acb: Received BeginTabletCopySession request for tablet 54cc03f6fe5a4bc9899442e38a6d95bb from peer 9144c65475bc402188177bb8a3fd3aa3 ({username='slave'} at 127.0.0.1:35708)
I20251209 03:05:11.982692 8351 tablet_copy_service.cc:161] P adf4c1aee58f48a2abd6612a80194acb: Beginning new tablet copy session on tablet 54cc03f6fe5a4bc9899442e38a6d95bb from peer 9144c65475bc402188177bb8a3fd3aa3 at {username='slave'} at 127.0.0.1:35708: session id = 9144c65475bc402188177bb8a3fd3aa3-54cc03f6fe5a4bc9899442e38a6d95bb
I20251209 03:05:11.983131 8351 tablet_copy_source_session.cc:215] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:05:11.983515 8595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54cc03f6fe5a4bc9899442e38a6d95bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:11.984498 8595 tablet_copy_client.cc:806] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Starting download of 0 data blocks...
I20251209 03:05:11.984607 8595 tablet_copy_client.cc:670] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Starting download of 1 WAL segments...
I20251209 03:05:11.985255 8595 tablet_copy_client.cc:538] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:05:11.985836 8595 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap starting.
I20251209 03:05:11.986824 8595 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:11.986965 8595 tablet_bootstrap.cc:492] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap complete.
I20251209 03:05:11.987046 8595 ts_tablet_manager.cc:1403] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:11.987154 8595 raft_consensus.cc:359] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:11.987211 8595 raft_consensus.cc:740] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9144c65475bc402188177bb8a3fd3aa3, State: Initialized, Role: LEARNER
I20251209 03:05:11.987238 8595 consensus_queue.cc:260] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:11.987330 8595 ts_tablet_manager.cc:1434] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:11.987552 8351 tablet_copy_service.cc:342] P adf4c1aee58f48a2abd6612a80194acb: Request end of tablet copy session 9144c65475bc402188177bb8a3fd3aa3-54cc03f6fe5a4bc9899442e38a6d95bb received from {username='slave'} at 127.0.0.1:35708
I20251209 03:05:11.987612 8351 tablet_copy_service.cc:434] P adf4c1aee58f48a2abd6612a80194acb: ending tablet copy session 9144c65475bc402188177bb8a3fd3aa3-54cc03f6fe5a4bc9899442e38a6d95bb on tablet 54cc03f6fe5a4bc9899442e38a6d95bb with peer 9144c65475bc402188177bb8a3fd3aa3
I20251209 03:05:12.223832 8416 raft_consensus.cc:1275] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20251209 03:05:12.224128 8588 consensus_queue.cc:1048] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251209 03:05:12.225562 8306 consensus_peers.cc:597] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:12.227082 8548 raft_consensus.cc:1217] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20251209 03:05:12.414000 8611 consensus_queue.cc:579] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Leader has been unable to successfully communicate with peer d864b254ec5d4cfd8453c7abb6a8f315 for more than 1 seconds (1.397s)
I20251209 03:05:12.433388 8588 consensus_queue.cc:579] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Leader has been unable to successfully communicate with peer d864b254ec5d4cfd8453c7abb6a8f315 for more than 1 seconds (1.031s)
I20251209 03:05:12.435938 8604 raft_consensus.cc:1064] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d: attempting to promote NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 to VOTER
I20251209 03:05:12.436133 8604 consensus_queue.cc:237] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } }
I20251209 03:05:12.436601 8548 raft_consensus.cc:1275] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:12.436712 8341 raft_consensus.cc:1275] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:12.436856 8611 consensus_queue.cc:1048] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:12.436954 8614 consensus_queue.cc:1048] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251209 03:05:12.437095 8381 consensus_peers.cc:597] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:12.437711 8611 raft_consensus.cc:2955] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.437857 8341 raft_consensus.cc:2955] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.438797 8548 raft_consensus.cc:2955] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.438880 8162 catalog_manager.cc:5654] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d reported cstate change: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8039d77cdb19445291bd2948b041d44d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:05:12.439962 8416 consensus_queue.cc:237] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } }
I20251209 03:05:12.440306 8341 raft_consensus.cc:1275] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:12.440306 8548 raft_consensus.cc:1275] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:12.440480 8614 consensus_queue.cc:1048] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:12.440577 8482 consensus_queue.cc:1048] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:12.441124 8611 raft_consensus.cc:2955] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.441273 8341 raft_consensus.cc:2955] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.441735 8548 raft_consensus.cc:2955] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.442091 8162 catalog_manager.cc:5654] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d reported cstate change: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New cstate: current_term: 1 leader_uuid: "8039d77cdb19445291bd2948b041d44d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:05:12.442494 8148 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 4eed7112e439482fab13e05b7998f41b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20251209 03:05:12.443337 8148 catalog_manager.cc:4712] TS d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4eed7112e439482fab13e05b7998f41b: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
I20251209 03:05:12.514465 8548 raft_consensus.cc:1217] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20251209 03:05:12.617156 8210 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver d864b254ec5d4cfd8453c7abb6a8f315 not available for placement
I20251209 03:05:12.730942 8482 consensus_queue.cc:579] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Leader has been unable to successfully communicate with peer d864b254ec5d4cfd8453c7abb6a8f315 for more than 1 seconds (1.430s)
I20251209 03:05:12.827327 8482 raft_consensus.cc:1064] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d: attempting to promote NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 to VOTER
I20251209 03:05:12.827528 8482 consensus_queue.cc:237] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } }
I20251209 03:05:12.828002 8548 raft_consensus.cc:1275] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:12.828002 8341 raft_consensus.cc:1275] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:12.828217 8611 consensus_queue.cc:1048] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251209 03:05:12.828321 8381 consensus_peers.cc:597] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:12.828328 8482 consensus_queue.cc:1048] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:12.829130 8611 raft_consensus.cc:2955] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.829257 8341 raft_consensus.cc:2955] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.830397 8548 raft_consensus.cc:2955] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.830475 8160 catalog_manager.cc:5654] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb reported cstate change: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8039d77cdb19445291bd2948b041d44d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.831524 8416 consensus_queue.cc:237] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } }
I20251209 03:05:12.831971 8341 raft_consensus.cc:1275] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:12.831979 8548 raft_consensus.cc:1275] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Refusing update from remote peer 8039d77cdb19445291bd2948b041d44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:12.832147 8614 consensus_queue.cc:1048] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:12.832248 8482 consensus_queue.cc:1048] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:12.832844 8482 raft_consensus.cc:2955] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.833017 8341 raft_consensus.cc:2955] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.833024 8548 raft_consensus.cc:2955] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:12.833961 8148 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 97ea2cd32b4742ce8fb65ca0b26be071 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:12.834013 8162 catalog_manager.cc:5654] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 reported cstate change: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New cstate: current_term: 1 leader_uuid: "8039d77cdb19445291bd2948b041d44d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.038780 8588 raft_consensus.cc:1064] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb: attempting to promote NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 to VOTER
I20251209 03:05:13.038982 8588 consensus_queue.cc:237] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } }
I20251209 03:05:13.039458 8548 raft_consensus.cc:1275] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 LEARNER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:13.039494 8416 raft_consensus.cc:1275] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:13.039770 8625 consensus_queue.cc:1048] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251209 03:05:13.039885 8306 consensus_peers.cc:597] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:13.039963 8628 consensus_queue.cc:1048] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:13.040566 8625 raft_consensus.cc:2955] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.040726 8416 raft_consensus.cc:2955] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.041642 8548 raft_consensus.cc:2955] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.041740 8162 catalog_manager.cc:5654] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d reported cstate change: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "adf4c1aee58f48a2abd6612a80194acb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.042559 8341 consensus_queue.cc:237] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } }
I20251209 03:05:13.042889 8548 raft_consensus.cc:1275] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:13.042995 8416 raft_consensus.cc:1275] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:13.043081 8588 consensus_queue.cc:1048] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:13.043231 8588 consensus_queue.cc:1048] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:13.043787 8627 raft_consensus.cc:2955] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.043910 8548 raft_consensus.cc:2955] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.043951 8416 raft_consensus.cc:2955] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.044705 8146 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 54cc03f6fe5a4bc9899442e38a6d95bb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:13.044834 8161 catalog_manager.cc:5654] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 reported cstate change: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New cstate: current_term: 1 leader_uuid: "adf4c1aee58f48a2abd6612a80194acb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:13.162237 8625 consensus_queue.cc:579] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Leader has been unable to successfully communicate with peer d864b254ec5d4cfd8453c7abb6a8f315 for more than 1 seconds (1.408s)
I20251209 03:05:13.163164 8341 consensus_queue.cc:237] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:13.163674 8416 raft_consensus.cc:1275] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20251209 03:05:13.163810 8304 consensus_peers.cc:597] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb -> Peer 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196:33957): Couldn't send request to peer 9144c65475bc402188177bb8a3fd3aa3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1cbd31efb904dfd83df937ad0fdfe5d. This is attempt 1: this message will repeat every 5th retry.
W20251209 03:05:13.163830 8306 consensus_peers.cc:597] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:13.163918 8628 consensus_queue.cc:1048] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:13.164474 8588 raft_consensus.cc:2955] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
I20251209 03:05:13.164638 8416 raft_consensus.cc:2955] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
I20251209 03:05:13.165441 8146 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c1cbd31efb904dfd83df937ad0fdfe5d with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251209 03:05:13.165654 8160 catalog_manager.cc:5654] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d reported cstate change: config changed from index -1 to 2, NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) added. New cstate: current_term: 2 leader_uuid: "adf4c1aee58f48a2abd6612a80194acb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } } }
W20251209 03:05:13.464076 8148 catalog_manager.cc:4712] TS d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 97ea2cd32b4742ce8fb65ca0b26be071: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
W20251209 03:05:13.617416 8210 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver d864b254ec5d4cfd8453c7abb6a8f315 not available for placement
I20251209 03:05:13.713311 8631 ts_tablet_manager.cc:933] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: Initiating tablet copy from peer adf4c1aee58f48a2abd6612a80194acb (127.2.170.194:44687)
I20251209 03:05:13.713483 8631 tablet_copy_client.cc:323] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:44687
I20251209 03:05:13.713735 8351 tablet_copy_service.cc:140] P adf4c1aee58f48a2abd6612a80194acb: Received BeginTabletCopySession request for tablet c1cbd31efb904dfd83df937ad0fdfe5d from peer 9144c65475bc402188177bb8a3fd3aa3 ({username='slave'} at 127.0.0.1:35708)
I20251209 03:05:13.713791 8351 tablet_copy_service.cc:161] P adf4c1aee58f48a2abd6612a80194acb: Beginning new tablet copy session on tablet c1cbd31efb904dfd83df937ad0fdfe5d from peer 9144c65475bc402188177bb8a3fd3aa3 at {username='slave'} at 127.0.0.1:35708: session id = 9144c65475bc402188177bb8a3fd3aa3-c1cbd31efb904dfd83df937ad0fdfe5d
I20251209 03:05:13.714215 8351 tablet_copy_source_session.cc:215] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:05:13.714519 8631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1cbd31efb904dfd83df937ad0fdfe5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:13.715495 8631 tablet_copy_client.cc:806] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Starting download of 0 data blocks...
I20251209 03:05:13.715615 8631 tablet_copy_client.cc:670] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Starting download of 1 WAL segments...
I20251209 03:05:13.716225 8631 tablet_copy_client.cc:538] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:05:13.716849 8631 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap starting.
I20251209 03:05:13.717911 8631 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:13.718050 8631 tablet_bootstrap.cc:492] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: Bootstrap complete.
I20251209 03:05:13.718132 8631 ts_tablet_manager.cc:1403] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:13.718240 8631 raft_consensus.cc:359] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:13.718297 8631 raft_consensus.cc:740] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 9144c65475bc402188177bb8a3fd3aa3, State: Initialized, Role: LEARNER
I20251209 03:05:13.718328 8631 consensus_queue.cc:260] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: true } }
I20251209 03:05:13.718473 8631 ts_tablet_manager.cc:1434] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251209 03:05:13.718667 8351 tablet_copy_service.cc:342] P adf4c1aee58f48a2abd6612a80194acb: Request end of tablet copy session 9144c65475bc402188177bb8a3fd3aa3-c1cbd31efb904dfd83df937ad0fdfe5d received from {username='slave'} at 127.0.0.1:35708
I20251209 03:05:13.718709 8351 tablet_copy_service.cc:434] P adf4c1aee58f48a2abd6612a80194acb: ending tablet copy session 9144c65475bc402188177bb8a3fd3aa3-c1cbd31efb904dfd83df937ad0fdfe5d on tablet c1cbd31efb904dfd83df937ad0fdfe5d with peer 9144c65475bc402188177bb8a3fd3aa3
I20251209 03:05:14.241349 8548 raft_consensus.cc:1217] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20251209 03:05:14.558455 8635 consensus_queue.cc:579] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Leader has been unable to successfully communicate with peer d864b254ec5d4cfd8453c7abb6a8f315 for more than 1 seconds (1.395s)
W20251209 03:05:14.617657 8210 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver d864b254ec5d4cfd8453c7abb6a8f315 not available for placement
W20251209 03:05:14.670418 8148 catalog_manager.cc:4712] TS d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4eed7112e439482fab13e05b7998f41b: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
I20251209 03:05:14.754325 8625 raft_consensus.cc:1064] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb: attempting to promote NON_VOTER 9144c65475bc402188177bb8a3fd3aa3 to VOTER
I20251209 03:05:14.754518 8625 consensus_queue.cc:237] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } }
I20251209 03:05:14.755000 8548 raft_consensus.cc:1275] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 LEARNER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251209 03:05:14.755003 8416 raft_consensus.cc:1275] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251209 03:05:14.755246 8642 consensus_queue.cc:1048] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:14.755419 8643 consensus_queue.cc:1048] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251209 03:05:14.755533 8306 consensus_peers.cc:597] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb -> Peer d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): Couldn't send request to peer d864b254ec5d4cfd8453c7abb6a8f315. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:14.756283 8635 raft_consensus.cc:2955] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:14.756419 8548 raft_consensus.cc:2955] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:14.757297 8416 raft_consensus.cc:2955] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:14.757404 8162 catalog_manager.cc:5654] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb reported cstate change: config changed from index 2 to 3, 9144c65475bc402188177bb8a3fd3aa3 (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "adf4c1aee58f48a2abd6612a80194acb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d864b254ec5d4cfd8453c7abb6a8f315" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 39167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:05:14.758626 8341 consensus_queue.cc:237] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } }
I20251209 03:05:14.758953 8416 raft_consensus.cc:1275] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251209 03:05:14.758975 8548 raft_consensus.cc:1275] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 FOLLOWER]: Refusing update from remote peer adf4c1aee58f48a2abd6612a80194acb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251209 03:05:14.759131 8635 consensus_queue.cc:1048] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:14.759208 8642 consensus_queue.cc:1048] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:14.759796 8642 raft_consensus.cc:2955] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:14.759985 8548 raft_consensus.cc:2955] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:14.759982 8416 raft_consensus.cc:2955] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } } }
I20251209 03:05:14.760673 8146 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet c1cbd31efb904dfd83df937ad0fdfe5d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:14.760901 8160 catalog_manager.cc:5654] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb reported cstate change: config changed from index 3 to 4, VOTER d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193) evicted. New cstate: current_term: 2 leader_uuid: "adf4c1aee58f48a2abd6612a80194acb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "adf4c1aee58f48a2abd6612a80194acb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 44687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8039d77cdb19445291bd2948b041d44d" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9144c65475bc402188177bb8a3fd3aa3" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 33957 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20251209 03:05:15.389701 8148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111) [suppressed 61 similar messages]
W20251209 03:05:15.930275 8148 catalog_manager.cc:4712] TS d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c1cbd31efb904dfd83df937ad0fdfe5d: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
W20251209 03:05:16.963356 8148 catalog_manager.cc:4712] TS d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c1cbd31efb904dfd83df937ad0fdfe5d: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
I20251209 03:05:18.035564 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:05:18.038753 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:18.038878 2731 tablet_replica.cc:333] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb: stopping tablet replica
I20251209 03:05:18.038939 2731 raft_consensus.cc:2243] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:05:18.039021 2731 raft_consensus.cc:2272] T c1cbd31efb904dfd83df937ad0fdfe5d P adf4c1aee58f48a2abd6612a80194acb [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.039197 2731 tablet_replica.cc:333] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb: stopping tablet replica
I20251209 03:05:18.039249 2731 raft_consensus.cc:2243] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:18.039300 2731 raft_consensus.cc:2272] T 4eed7112e439482fab13e05b7998f41b P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.039409 2731 tablet_replica.cc:333] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb: stopping tablet replica
I20251209 03:05:18.039444 2731 raft_consensus.cc:2243] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:18.039495 2731 raft_consensus.cc:2272] T 54cc03f6fe5a4bc9899442e38a6d95bb P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.039606 2731 tablet_replica.cc:333] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb: stopping tablet replica
I20251209 03:05:18.039633 2731 raft_consensus.cc:2243] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:18.039677 2731 raft_consensus.cc:2272] T 97ea2cd32b4742ce8fb65ca0b26be071 P adf4c1aee58f48a2abd6612a80194acb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.051388 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:05:18.052480 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:05:18.055656 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:18.055811 2731 tablet_replica.cc:333] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d: stopping tablet replica
I20251209 03:05:18.055883 2731 raft_consensus.cc:2243] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:18.055958 2731 raft_consensus.cc:2272] T 4eed7112e439482fab13e05b7998f41b P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.056113 2731 tablet_replica.cc:333] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d: stopping tablet replica
I20251209 03:05:18.056150 2731 raft_consensus.cc:2243] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:18.056200 2731 raft_consensus.cc:2272] T c1cbd31efb904dfd83df937ad0fdfe5d P 8039d77cdb19445291bd2948b041d44d [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.056314 2731 tablet_replica.cc:333] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d: stopping tablet replica
I20251209 03:05:18.056346 2731 raft_consensus.cc:2243] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:18.056385 2731 raft_consensus.cc:2272] T 54cc03f6fe5a4bc9899442e38a6d95bb P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.056504 2731 tablet_replica.cc:333] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d: stopping tablet replica
I20251209 03:05:18.056536 2731 raft_consensus.cc:2243] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:18.056584 2731 raft_consensus.cc:2272] T 97ea2cd32b4742ce8fb65ca0b26be071 P 8039d77cdb19445291bd2948b041d44d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.068733 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:05:18.069769 2731 tablet_server.cc:178] TabletServer@127.2.170.196:0 shutting down...
I20251209 03:05:18.072759 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:18.072912 2731 tablet_replica.cc:333] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3: stopping tablet replica
I20251209 03:05:18.073007 2731 raft_consensus.cc:2243] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:18.073060 2731 raft_consensus.cc:2272] T c1cbd31efb904dfd83df937ad0fdfe5d P 9144c65475bc402188177bb8a3fd3aa3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.073212 2731 tablet_replica.cc:333] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3: stopping tablet replica
I20251209 03:05:18.073247 2731 raft_consensus.cc:2243] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:18.073287 2731 raft_consensus.cc:2272] T 4eed7112e439482fab13e05b7998f41b P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.073395 2731 tablet_replica.cc:333] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3: stopping tablet replica
I20251209 03:05:18.073422 2731 raft_consensus.cc:2243] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:18.073460 2731 raft_consensus.cc:2272] T 54cc03f6fe5a4bc9899442e38a6d95bb P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.073565 2731 tablet_replica.cc:333] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3: stopping tablet replica
I20251209 03:05:18.073598 2731 raft_consensus.cc:2243] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:18.073638 2731 raft_consensus.cc:2272] T 97ea2cd32b4742ce8fb65ca0b26be071 P 9144c65475bc402188177bb8a3fd3aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:18.085078 2731 tablet_server.cc:195] TabletServer@127.2.170.196:0 shutdown complete.
I20251209 03:05:18.086479 2731 master.cc:561] Master@127.2.170.254:41003 shutting down...
W20251209 03:05:19.029016 8148 catalog_manager.cc:4712] TS d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c1cbd31efb904dfd83df937ad0fdfe5d: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
W20251209 03:05:20.884994 8148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20251209 03:05:20.885442 8148 catalog_manager.cc:4712] TS d864b254ec5d4cfd8453c7abb6a8f315 (127.2.170.193:39167): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 4eed7112e439482fab13e05b7998f41b: Network error: Client connection negotiation failed: client connection to 127.2.170.193:39167: connect: Connection refused (error 111)
I20251209 03:05:21.679754 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:21.679893 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:21.679913 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P 88d79c456f554476aeb296b43979728d: stopping tablet replica
I20251209 03:05:21.691658 2731 master.cc:583] Master@127.2.170.254:41003 shutdown complete.
[ OK ] AutoRebalancerTest.NoRebalancingIfReplicasRecovering (13104 ms)
[ RUN ] AutoRebalancerTest.TestHandlingFailedTservers
I20251209 03:05:21.697530 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:34021
I20251209 03:05:21.697716 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:21.698880 8659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:21.698899 8662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:21.699031 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:21.699193 8660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:21.699308 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:21.699338 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:21.699352 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249521699353 us; error 0 us; skew 500 ppm
I20251209 03:05:21.699882 2731 webserver.cc:492] Webserver started at http://127.2.170.254:37297/ using document root <none> and password file <none>
I20251209 03:05:21.699944 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:21.699976 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:21.700018 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:21.700279 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "c18f8c6c2b864102a93a5e7692650c5e"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:21.701046 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:21.701442 8667 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.701558 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.701601 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "c18f8c6c2b864102a93a5e7692650c5e"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:21.701639 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:21.714999 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:21.715169 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:21.717494 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:34021
I20251209 03:05:21.719982 8719 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:34021 every 8 connection(s)
I20251209 03:05:21.720109 8720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:21.721091 8720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e: Bootstrap starting.
I20251209 03:05:21.721329 8720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:21.721793 8720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e: No bootstrap required, opened a new log
I20251209 03:05:21.721905 8720 raft_consensus.cc:359] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18f8c6c2b864102a93a5e7692650c5e" member_type: VOTER }
I20251209 03:05:21.721951 8720 raft_consensus.cc:385] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:21.721963 8720 raft_consensus.cc:740] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c18f8c6c2b864102a93a5e7692650c5e, State: Initialized, Role: FOLLOWER
I20251209 03:05:21.721997 8720 consensus_queue.cc:260] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18f8c6c2b864102a93a5e7692650c5e" member_type: VOTER }
I20251209 03:05:21.722030 8720 raft_consensus.cc:399] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:05:21.722048 8720 raft_consensus.cc:493] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:05:21.722065 8720 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:21.722497 8720 raft_consensus.cc:515] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18f8c6c2b864102a93a5e7692650c5e" member_type: VOTER }
I20251209 03:05:21.722548 8720 leader_election.cc:304] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c18f8c6c2b864102a93a5e7692650c5e; no voters:
I20251209 03:05:21.722626 8720 leader_election.cc:290] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:05:21.722681 8723 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:21.722751 8720 sys_catalog.cc:565] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:05:21.722827 8723 raft_consensus.cc:697] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 1 LEADER]: Becoming Leader. State: Replica: c18f8c6c2b864102a93a5e7692650c5e, State: Running, Role: LEADER
I20251209 03:05:21.722883 8723 consensus_queue.cc:237] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18f8c6c2b864102a93a5e7692650c5e" member_type: VOTER }
I20251209 03:05:21.723088 8724 sys_catalog.cc:455] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c18f8c6c2b864102a93a5e7692650c5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18f8c6c2b864102a93a5e7692650c5e" member_type: VOTER } }
I20251209 03:05:21.723120 8725 sys_catalog.cc:455] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c18f8c6c2b864102a93a5e7692650c5e. Latest consensus state: current_term: 1 leader_uuid: "c18f8c6c2b864102a93a5e7692650c5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18f8c6c2b864102a93a5e7692650c5e" member_type: VOTER } }
I20251209 03:05:21.723198 8724 sys_catalog.cc:458] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [sys.catalog]: This master's current role is: LEADER
I20251209 03:05:21.723232 8725 sys_catalog.cc:458] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [sys.catalog]: This master's current role is: LEADER
I20251209 03:05:21.723454 8730 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:05:21.723579 8730 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:05:21.723961 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:05:21.724134 8730 catalog_manager.cc:1357] Generated new cluster ID: e360ea85f75a4096809fdcabaa34fbc9
I20251209 03:05:21.724176 8730 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:05:21.735769 8730 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:05:21.736196 8730 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:05:21.739916 8730 catalog_manager.cc:6027] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e: Generated new TSK 0
I20251209 03:05:21.740015 8730 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:05:21.755734 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:21.757011 8741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:21.757122 8744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:21.757213 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:21.757122 8742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:21.757417 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:21.757450 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:21.757462 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249521757462 us; error 0 us; skew 500 ppm
I20251209 03:05:21.758038 2731 webserver.cc:492] Webserver started at http://127.2.170.193:40439/ using document root <none> and password file <none>
I20251209 03:05:21.758109 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:21.758142 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:21.758174 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:21.758404 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "b082d2a61d134d2eb84934cc60a3c82a"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:21.759161 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:21.759546 8749 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.759666 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.759707 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "b082d2a61d134d2eb84934cc60a3c82a"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:21.759745 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:21.768496 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:21.768679 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:21.768805 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:21.768978 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:21.769004 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.769030 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:21.769055 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.771633 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:45767
I20251209 03:05:21.771690 8812 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:45767 every 8 connection(s)
I20251209 03:05:21.771975 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:21.773073 8820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:21.773173 8818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:21.773516 8817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:21.773545 2731 server_base.cc:1047] running on GCE node
I20251209 03:05:21.773651 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:21.773715 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:21.773733 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249521773733 us; error 0 us; skew 500 ppm
I20251209 03:05:21.774293 2731 webserver.cc:492] Webserver started at http://127.2.170.194:34049/ using document root <none> and password file <none>
I20251209 03:05:21.774369 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:21.774413 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:21.774457 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:21.774751 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "e40301b8c55143d395ca013de36123cb"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:21.775539 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:21.775902 8825 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.776005 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.776043 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "e40301b8c55143d395ca013de36123cb"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:21.776084 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:21.781284 8813 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34021
I20251209 03:05:21.781365 8813 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:21.781484 8813 heartbeater.cc:507] Master 127.2.170.254:34021 requested a full tablet report, sending...
I20251209 03:05:21.781752 8684 ts_manager.cc:194] Registered new tserver with Master: b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767)
I20251209 03:05:21.782560 8684 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50798
I20251209 03:05:21.808521 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:21.808740 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:21.808876 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:21.809089 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:21.809118 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.809140 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:21.809163 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.811806 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:46217
I20251209 03:05:21.811856 8888 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:46217 every 8 connection(s)
I20251209 03:05:21.812148 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:21.813336 8892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:21.813467 8895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:21.813621 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:21.813575 8893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:21.813843 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:21.813886 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:21.813903 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249521813903 us; error 0 us; skew 500 ppm
I20251209 03:05:21.814648 2731 webserver.cc:492] Webserver started at http://127.2.170.195:43751/ using document root <none> and password file <none>
I20251209 03:05:21.814749 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:21.814797 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:21.814846 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:21.815058 8889 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34021
I20251209 03:05:21.815120 8889 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:21.815120 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "edb756e223574530abf803c8f40e072b"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:21.815241 8889 heartbeater.cc:507] Master 127.2.170.254:34021 requested a full tablet report, sending...
I20251209 03:05:21.815469 8684 ts_manager.cc:194] Registered new tserver with Master: e40301b8c55143d395ca013de36123cb (127.2.170.194:46217)
I20251209 03:05:21.815800 8684 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50812
I20251209 03:05:21.816020 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:21.816464 8900 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.816576 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.816619 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "edb756e223574530abf803c8f40e072b"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:21.816661 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:21.826799 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:21.827028 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:21.827169 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:21.827359 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:21.827387 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.827409 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:21.827433 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:21.830303 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:34053
I20251209 03:05:21.830358 8963 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:34053 every 8 connection(s)
I20251209 03:05:21.833245 8964 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34021
I20251209 03:05:21.833302 8964 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:21.833428 8964 heartbeater.cc:507] Master 127.2.170.254:34021 requested a full tablet report, sending...
I20251209 03:05:21.833640 8684 ts_manager.cc:194] Registered new tserver with Master: edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:21.833781 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003302854s
I20251209 03:05:21.833998 8684 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50826
I20251209 03:05:22.783092 8813 heartbeater.cc:499] Master 127.2.170.254:34021 was elected leader, sending a full tablet report...
I20251209 03:05:22.816344 8889 heartbeater.cc:499] Master 127.2.170.254:34021 was elected leader, sending a full tablet report...
I20251209 03:05:22.834432 8964 heartbeater.cc:499] Master 127.2.170.254:34021 was elected leader, sending a full tablet report...
I20251209 03:05:22.858748 2731 test_util.cc:276] Using random seed: 784333561
I20251209 03:05:22.863487 8684 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50840:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:05:22.863651 8684 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:05:22.868552 8775 tablet_service.cc:1505] Processing CreateTablet for tablet 01d23b8eaeea4bc983d0c12df54ff83f (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:22.868518 8776 tablet_service.cc:1505] Processing CreateTablet for tablet 905884ffefc9486c8fc0a8fac0fb6b98 (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:22.868664 8776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 905884ffefc9486c8fc0a8fac0fb6b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.868691 8774 tablet_service.cc:1505] Processing CreateTablet for tablet 2f662583a5c641ab946b9fb216fea5fa (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:22.868796 8774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f662583a5c641ab946b9fb216fea5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.868976 8853 tablet_service.cc:1505] Processing CreateTablet for tablet 77cdfff2712448f280f34a91ead8a040 (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:22.869092 8853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77cdfff2712448f280f34a91ead8a040. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.868518 8777 tablet_service.cc:1505] Processing CreateTablet for tablet 77cdfff2712448f280f34a91ead8a040 (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:22.870090 8777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77cdfff2712448f280f34a91ead8a040. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.870136 8984 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap starting.
I20251209 03:05:22.870409 8985 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Bootstrap starting.
I20251209 03:05:22.870440 8852 tablet_service.cc:1505] Processing CreateTablet for tablet 905884ffefc9486c8fc0a8fac0fb6b98 (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:22.870523 8852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 905884ffefc9486c8fc0a8fac0fb6b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.870532 8984 tablet_bootstrap.cc:654] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.870822 8985 tablet_bootstrap.cc:654] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.871351 8775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d23b8eaeea4bc983d0c12df54ff83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.871426 8985 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: No bootstrap required, opened a new log
I20251209 03:05:22.871459 8985 ts_tablet_manager.cc:1403] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:22.871575 8851 tablet_service.cc:1505] Processing CreateTablet for tablet 01d23b8eaeea4bc983d0c12df54ff83f (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:22.871603 8985 raft_consensus.cc:359] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:22.871670 8985 raft_consensus.cc:385] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.871639 8851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d23b8eaeea4bc983d0c12df54ff83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.871692 8850 tablet_service.cc:1505] Processing CreateTablet for tablet 2f662583a5c641ab946b9fb216fea5fa (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:22.871796 8850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f662583a5c641ab946b9fb216fea5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.872990 8928 tablet_service.cc:1505] Processing CreateTablet for tablet 77cdfff2712448f280f34a91ead8a040 (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:22.873042 8926 tablet_service.cc:1505] Processing CreateTablet for tablet 01d23b8eaeea4bc983d0c12df54ff83f (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:22.873101 8928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77cdfff2712448f280f34a91ead8a040. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.873143 8925 tablet_service.cc:1505] Processing CreateTablet for tablet 2f662583a5c641ab946b9fb216fea5fa (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:22.873229 8925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f662583a5c641ab946b9fb216fea5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.873548 8984 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: No bootstrap required, opened a new log
I20251209 03:05:22.873586 8984 ts_tablet_manager.cc:1403] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251209 03:05:22.873752 8926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d23b8eaeea4bc983d0c12df54ff83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.873723 8984 raft_consensus.cc:359] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.873800 8984 raft_consensus.cc:385] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.873819 8984 raft_consensus.cc:740] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.873857 8984 consensus_queue.cc:260] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.873989 8984 ts_tablet_manager.cc:1434] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:22.874043 8984 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap starting.
I20251209 03:05:22.874395 8984 tablet_bootstrap.cc:654] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.874953 8984 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: No bootstrap required, opened a new log
I20251209 03:05:22.874990 8984 ts_tablet_manager.cc:1403] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:22.873004 8927 tablet_service.cc:1505] Processing CreateTablet for tablet 905884ffefc9486c8fc0a8fac0fb6b98 (DEFAULT_TABLE table=test-workload [id=05b20a28b651405ab3876d42405b45fe]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:22.875118 8927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 905884ffefc9486c8fc0a8fac0fb6b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:22.875118 8984 raft_consensus.cc:359] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:22.874832 8989 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Bootstrap starting.
I20251209 03:05:22.875217 8984 raft_consensus.cc:385] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.875240 8984 raft_consensus.cc:740] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.875286 8984 consensus_queue.cc:260] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:22.875391 8984 ts_tablet_manager.cc:1434] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:22.875442 8984 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap starting.
I20251209 03:05:22.875598 8989 tablet_bootstrap.cc:654] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.875727 8984 tablet_bootstrap.cc:654] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.876127 8989 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: No bootstrap required, opened a new log
I20251209 03:05:22.876163 8989 ts_tablet_manager.cc:1403] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:22.876283 8989 raft_consensus.cc:359] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:22.876338 8989 raft_consensus.cc:385] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.876356 8989 raft_consensus.cc:740] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.876415 8989 consensus_queue.cc:260] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:22.876557 8989 ts_tablet_manager.cc:1434] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:22.876617 8989 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Bootstrap starting.
I20251209 03:05:22.871690 8985 raft_consensus.cc:740] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40301b8c55143d395ca013de36123cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.876965 8989 tablet_bootstrap.cc:654] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.877058 8985 consensus_queue.cc:260] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:22.877321 8985 ts_tablet_manager.cc:1434] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251209 03:05:22.877424 8985 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Bootstrap starting.
I20251209 03:05:22.877588 8984 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: No bootstrap required, opened a new log
I20251209 03:05:22.877642 8984 ts_tablet_manager.cc:1403] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:22.877701 8989 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: No bootstrap required, opened a new log
I20251209 03:05:22.877736 8989 ts_tablet_manager.cc:1403] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:22.877805 8984 raft_consensus.cc:359] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.877835 8985 tablet_bootstrap.cc:654] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.877868 8984 raft_consensus.cc:385] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.877851 8989 raft_consensus.cc:359] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.877900 8984 raft_consensus.cc:740] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.877912 8989 raft_consensus.cc:385] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.877931 8989 raft_consensus.cc:740] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.877945 8984 consensus_queue.cc:260] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.877965 8989 consensus_queue.cc:260] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.878021 8984 ts_tablet_manager.cc:1434] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:22.878031 8989 ts_tablet_manager.cc:1434] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:22.878083 8984 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap starting.
I20251209 03:05:22.878098 8989 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Bootstrap starting.
I20251209 03:05:22.878458 8984 tablet_bootstrap.cc:654] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.878463 8989 tablet_bootstrap.cc:654] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.878521 8985 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: No bootstrap required, opened a new log
I20251209 03:05:22.878562 8985 ts_tablet_manager.cc:1403] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:22.878715 8985 raft_consensus.cc:359] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.878759 8985 raft_consensus.cc:385] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.878777 8985 raft_consensus.cc:740] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40301b8c55143d395ca013de36123cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.878816 8985 consensus_queue.cc:260] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.878890 8985 ts_tablet_manager.cc:1434] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:22.878937 8985 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Bootstrap starting.
I20251209 03:05:22.879012 8984 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: No bootstrap required, opened a new log
I20251209 03:05:22.879053 8984 ts_tablet_manager.cc:1403] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:22.879187 8984 raft_consensus.cc:359] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.879232 8984 raft_consensus.cc:385] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.879249 8984 raft_consensus.cc:740] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.879297 8985 tablet_bootstrap.cc:654] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.879616 8989 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: No bootstrap required, opened a new log
I20251209 03:05:22.879657 8989 ts_tablet_manager.cc:1403] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:22.879284 8984 consensus_queue.cc:260] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.879747 8984 ts_tablet_manager.cc:1434] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:22.879807 8989 raft_consensus.cc:359] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.879861 8989 raft_consensus.cc:385] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.879815 8988 raft_consensus.cc:493] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:22.879879 8989 raft_consensus.cc:740] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.879920 8988 raft_consensus.cc:515] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.879999 8989 consensus_queue.cc:260] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.880072 8988 leader_election.cc:290] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e40301b8c55143d395ca013de36123cb (127.2.170.194:46217), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:22.880093 8989 ts_tablet_manager.cc:1434] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:22.880156 8989 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Bootstrap starting.
I20251209 03:05:22.880558 8989 tablet_bootstrap.cc:654] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.880750 8985 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: No bootstrap required, opened a new log
I20251209 03:05:22.880795 8985 ts_tablet_manager.cc:1403] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:22.880926 8985 raft_consensus.cc:359] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.880977 8985 raft_consensus.cc:385] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.880995 8985 raft_consensus.cc:740] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40301b8c55143d395ca013de36123cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.881031 8985 consensus_queue.cc:260] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.881105 8985 ts_tablet_manager.cc:1434] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:22.881155 8985 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Bootstrap starting.
I20251209 03:05:22.881526 8985 tablet_bootstrap.cc:654] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:22.882738 8985 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: No bootstrap required, opened a new log
I20251209 03:05:22.882793 8985 ts_tablet_manager.cc:1403] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251209 03:05:22.882956 8985 raft_consensus.cc:359] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.883006 8985 raft_consensus.cc:385] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.883025 8985 raft_consensus.cc:740] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40301b8c55143d395ca013de36123cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.883061 8985 consensus_queue.cc:260] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.883131 8985 ts_tablet_manager.cc:1434] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:22.883325 8863 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d23b8eaeea4bc983d0c12df54ff83f" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb" is_pre_election: true
I20251209 03:05:22.883389 8863 raft_consensus.cc:2468] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 0.
I20251209 03:05:22.883526 8751 leader_election.cc:304] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, e40301b8c55143d395ca013de36123cb; no voters:
I20251209 03:05:22.883649 8989 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: No bootstrap required, opened a new log
I20251209 03:05:22.883697 8989 ts_tablet_manager.cc:1403] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251209 03:05:22.883852 8989 raft_consensus.cc:359] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.883903 8989 raft_consensus.cc:385] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:22.883922 8989 raft_consensus.cc:740] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Initialized, Role: FOLLOWER
I20251209 03:05:22.883960 8989 consensus_queue.cc:260] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.884289 8989 ts_tablet_manager.cc:1434] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:22.883697 9000 raft_consensus.cc:2804] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:22.884433 9000 raft_consensus.cc:493] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:22.884467 9000 raft_consensus.cc:3060] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:22.884639 8938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d23b8eaeea4bc983d0c12df54ff83f" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:22.884699 8938 raft_consensus.cc:2468] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 0.
I20251209 03:05:22.885169 9000 raft_consensus.cc:515] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.885268 9000 leader_election.cc:290] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 1 election: Requested vote from peers e40301b8c55143d395ca013de36123cb (127.2.170.194:46217), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:22.885372 8863 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d23b8eaeea4bc983d0c12df54ff83f" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb"
I20251209 03:05:22.885391 8938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d23b8eaeea4bc983d0c12df54ff83f" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:22.885417 8863 raft_consensus.cc:3060] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:22.885449 8938 raft_consensus.cc:3060] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:22.886050 8863 raft_consensus.cc:2468] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 1.
I20251209 03:05:22.886065 8938 raft_consensus.cc:2468] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 1.
I20251209 03:05:22.886220 8751 leader_election.cc:304] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, e40301b8c55143d395ca013de36123cb; no voters:
I20251209 03:05:22.886289 9000 raft_consensus.cc:2804] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:22.886344 9000 raft_consensus.cc:697] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 LEADER]: Becoming Leader. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Running, Role: LEADER
I20251209 03:05:22.886376 9000 consensus_queue.cc:237] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:22.886827 8683 catalog_manager.cc:5654] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a reported cstate change: term changed from 0 to 1, leader changed from <none> to b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193). New cstate: current_term: 1 leader_uuid: "b082d2a61d134d2eb84934cc60a3c82a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } }
W20251209 03:05:23.022796 8814 tablet.cc:2378] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251209 03:05:23.071686 8994 raft_consensus.cc:493] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:23.071786 8994 raft_consensus.cc:515] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:23.071950 8994 leader_election.cc:290] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:23.075474 8938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:23.075498 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a" is_pre_election: true
I20251209 03:05:23.075572 8938 raft_consensus.cc:2468] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 0.
I20251209 03:05:23.075625 8787 raft_consensus.cc:2468] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 0.
I20251209 03:05:23.075745 8827 leader_election.cc:304] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:23.075834 8994 raft_consensus.cc:2804] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:23.075883 8994 raft_consensus.cc:493] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:23.075907 8994 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.078372 8994 raft_consensus.cc:515] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:23.078500 8994 leader_election.cc:290] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 election: Requested vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:23.078665 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a"
I20251209 03:05:23.078665 8938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:23.078769 8787 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.078774 8938 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.079429 8938 raft_consensus.cc:2468] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 1.
I20251209 03:05:23.079429 8787 raft_consensus.cc:2468] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 1.
I20251209 03:05:23.079627 8827 leader_election.cc:304] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:23.079699 8994 raft_consensus.cc:2804] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:23.079831 8994 raft_consensus.cc:697] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: Becoming Leader. State: Replica: e40301b8c55143d395ca013de36123cb, State: Running, Role: LEADER
I20251209 03:05:23.079882 8994 consensus_queue.cc:237] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:23.080382 8683 catalog_manager.cc:5654] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb reported cstate change: term changed from 0 to 1, leader changed from <none> to e40301b8c55143d395ca013de36123cb (127.2.170.194). New cstate: current_term: 1 leader_uuid: "e40301b8c55143d395ca013de36123cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:23.158967 8994 raft_consensus.cc:493] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:23.159070 8994 raft_consensus.cc:515] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:23.159207 8994 leader_election.cc:290] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:23.159449 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a" is_pre_election: true
I20251209 03:05:23.159449 8938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:23.159562 8787 raft_consensus.cc:2468] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 0.
I20251209 03:05:23.159567 8938 raft_consensus.cc:2468] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 0.
I20251209 03:05:23.159741 8827 leader_election.cc:304] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:23.159859 8994 raft_consensus.cc:2804] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:23.159920 8994 raft_consensus.cc:493] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:23.159940 8994 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.160558 8994 raft_consensus.cc:515] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:23.160686 8994 leader_election.cc:290] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 election: Requested vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:23.160806 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a"
I20251209 03:05:23.160815 8938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:23.160882 8787 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.160884 8938 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.161509 8787 raft_consensus.cc:2468] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 1.
I20251209 03:05:23.161509 8938 raft_consensus.cc:2468] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 1.
I20251209 03:05:23.161718 8827 leader_election.cc:304] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:23.161808 8994 raft_consensus.cc:2804] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:23.161864 8994 raft_consensus.cc:697] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: Becoming Leader. State: Replica: e40301b8c55143d395ca013de36123cb, State: Running, Role: LEADER
I20251209 03:05:23.161906 8994 consensus_queue.cc:237] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:23.162454 8683 catalog_manager.cc:5654] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb reported cstate change: term changed from 0 to 1, leader changed from <none> to e40301b8c55143d395ca013de36123cb (127.2.170.194). New cstate: current_term: 1 leader_uuid: "e40301b8c55143d395ca013de36123cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:23.238307 8994 raft_consensus.cc:493] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:23.238425 8994 raft_consensus.cc:515] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:23.238579 8994 leader_election.cc:290] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edb756e223574530abf803c8f40e072b (127.2.170.195:34053), b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767)
I20251209 03:05:23.238824 8938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:23.238834 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a" is_pre_election: true
I20251209 03:05:23.238933 8787 raft_consensus.cc:2468] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 0.
I20251209 03:05:23.238933 8938 raft_consensus.cc:2468] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 0.
I20251209 03:05:23.239079 8827 leader_election.cc:304] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, e40301b8c55143d395ca013de36123cb; no voters:
I20251209 03:05:23.239176 8994 raft_consensus.cc:2804] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:23.239248 8994 raft_consensus.cc:493] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:23.239262 8994 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.239878 8994 raft_consensus.cc:515] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:23.239982 8994 leader_election.cc:290] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 election: Requested vote from peers edb756e223574530abf803c8f40e072b (127.2.170.195:34053), b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767)
I20251209 03:05:23.240139 8938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:23.240198 8938 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.240173 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a"
I20251209 03:05:23.240239 8787 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:23.240717 8938 raft_consensus.cc:2468] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 1.
I20251209 03:05:23.240841 8787 raft_consensus.cc:2468] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 1.
I20251209 03:05:23.240891 8827 leader_election.cc:304] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:23.240983 8994 raft_consensus.cc:2804] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:23.241032 8994 raft_consensus.cc:697] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: Becoming Leader. State: Replica: e40301b8c55143d395ca013de36123cb, State: Running, Role: LEADER
I20251209 03:05:23.241081 8994 consensus_queue.cc:237] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:23.241592 8683 catalog_manager.cc:5654] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb reported cstate change: term changed from 0 to 1, leader changed from <none> to e40301b8c55143d395ca013de36123cb (127.2.170.194). New cstate: current_term: 1 leader_uuid: "e40301b8c55143d395ca013de36123cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:23.309944 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:05:23.312273 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:23.312446 2731 tablet_replica.cc:333] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: stopping tablet replica
I20251209 03:05:23.312508 2731 raft_consensus.cc:2243] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:23.312556 2731 raft_consensus.cc:2272] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.312767 2731 tablet_replica.cc:333] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: stopping tablet replica
I20251209 03:05:23.312808 2731 raft_consensus.cc:2243] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:23.312836 2731 raft_consensus.cc:2272] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.313043 2731 tablet_replica.cc:333] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: stopping tablet replica
I20251209 03:05:23.313086 2731 raft_consensus.cc:2243] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:23.313124 2731 pending_rounds.cc:70] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Trying to abort 1 pending ops.
I20251209 03:05:23.313148 2731 pending_rounds.cc:77] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7230462045742993408 op_type: NO_OP noop_request { }
I20251209 03:05:23.313194 2731 raft_consensus.cc:2889] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251209 03:05:23.313220 2731 raft_consensus.cc:2272] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.313452 2731 tablet_replica.cc:333] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: stopping tablet replica
I20251209 03:05:23.313503 2731 raft_consensus.cc:2243] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:23.313537 2731 raft_consensus.cc:2272] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.315308 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
I20251209 03:05:23.316309 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:05:23.318789 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:23.318933 2731 tablet_replica.cc:333] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: stopping tablet replica
I20251209 03:05:23.318985 2731 raft_consensus.cc:2243] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:23.319038 2731 pending_rounds.cc:70] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Trying to abort 1 pending ops.
I20251209 03:05:23.319062 2731 pending_rounds.cc:77] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7230462046535659520 op_type: NO_OP noop_request { }
I20251209 03:05:23.319113 2731 raft_consensus.cc:2889] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251209 03:05:23.319150 2731 raft_consensus.cc:2272] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.319404 2731 tablet_replica.cc:333] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: stopping tablet replica
I20251209 03:05:23.319447 2731 raft_consensus.cc:2243] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:23.319475 2731 raft_consensus.cc:2272] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.319623 2731 tablet_replica.cc:333] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: stopping tablet replica
I20251209 03:05:23.319657 2731 raft_consensus.cc:2243] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:23.319691 2731 pending_rounds.cc:70] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Trying to abort 1 pending ops.
I20251209 03:05:23.319715 2731 pending_rounds.cc:77] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7230462047195918336 op_type: NO_OP noop_request { }
I20251209 03:05:23.319756 2731 raft_consensus.cc:2889] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251209 03:05:23.319780 2731 raft_consensus.cc:2272] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.319979 2731 tablet_replica.cc:333] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: stopping tablet replica
I20251209 03:05:23.320022 2731 raft_consensus.cc:2243] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:23.320061 2731 pending_rounds.cc:70] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Trying to abort 1 pending ops.
I20251209 03:05:23.320084 2731 pending_rounds.cc:77] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7230462046871605248 op_type: NO_OP noop_request { }
I20251209 03:05:23.320120 2731 raft_consensus.cc:2889] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251209 03:05:23.320147 2731 raft_consensus.cc:2272] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.331867 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:05:23.332799 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:05:23.334894 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:23.335022 2731 tablet_replica.cc:333] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: stopping tablet replica
I20251209 03:05:23.335085 2731 raft_consensus.cc:2243] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:23.335129 2731 raft_consensus.cc:2272] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.335310 2731 tablet_replica.cc:333] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: stopping tablet replica
I20251209 03:05:23.335348 2731 raft_consensus.cc:2243] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:23.335374 2731 raft_consensus.cc:2272] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.335505 2731 tablet_replica.cc:333] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: stopping tablet replica
I20251209 03:05:23.335536 2731 raft_consensus.cc:2243] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:23.335558 2731 raft_consensus.cc:2272] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.335697 2731 tablet_replica.cc:333] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: stopping tablet replica
I20251209 03:05:23.335733 2731 raft_consensus.cc:2243] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:23.335757 2731 raft_consensus.cc:2272] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:23.347124 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:05:23.348140 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:23.349232 9012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:23.349378 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:23.349395 9013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:23.349473 9015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:23.349586 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:23.349637 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:23.349665 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249523349665 us; error 0 us; skew 500 ppm
I20251209 03:05:23.352912 2731 webserver.cc:492] Webserver started at http://127.2.170.196:42311/ using document root <none> and password file <none>
I20251209 03:05:23.352999 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:23.353051 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:23.353103 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:23.353444 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-3-root/instance:
uuid: "c70df044711546178be1fc7bf0f77dfd"
format_stamp: "Formatted at 2025-12-09 03:05:23 on dist-test-slave-4cb0"
I20251209 03:05:23.354346 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:23.354830 9020 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:23.354946 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251209 03:05:23.354996 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-3-root
uuid: "c70df044711546178be1fc7bf0f77dfd"
format_stamp: "Formatted at 2025-12-09 03:05:23 on dist-test-slave-4cb0"
I20251209 03:05:23.355037 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:23.376905 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:23.377074 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:23.377177 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:23.377312 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:23.377337 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:23.377359 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:23.377377 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:23.379887 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.196:41917
I20251209 03:05:23.381592 9083 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.196:41917 every 8 connection(s)
I20251209 03:05:23.384326 9084 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34021
I20251209 03:05:23.384393 9084 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:23.384490 9084 heartbeater.cc:507] Master 127.2.170.254:34021 requested a full tablet report, sending...
I20251209 03:05:23.384708 8683 ts_manager.cc:194] Registered new tserver with Master: c70df044711546178be1fc7bf0f77dfd (127.2.170.196:41917)
I20251209 03:05:23.385123 8683 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50852
W20251209 03:05:23.724572 8733 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:23.725412 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:23.725473 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:23.726030 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:23.726081 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:23.726599 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:23.726652 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
I20251209 03:05:24.385605 9084 heartbeater.cc:499] Master 127.2.170.254:34021 was elected leader, sending a full tablet report...
W20251209 03:05:24.727480 8733 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:24.728158 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:24.728205 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:24.728608 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:24.728675 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:24.729048 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:24.729094 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:25.729981 8733 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:25.730700 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:25.730752 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:25.731158 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:25.731204 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:25.731612 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:25.731662 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:26.732602 8733 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:26.733006 8669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20251209 03:05:26.733441 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:26.733484 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:26.733924 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:26.733968 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:26.734344 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:26.734385 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:27.735252 8733 auto_rebalancer.cc:253] failed to send replica move request: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:27.735985 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:27.736032 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.193:45767: connect: Connection refused (error 111)
W20251209 03:05:27.736532 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:27.736577 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:27.736989 8733 auto_rebalancer.cc:663] Could not move replica: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:27.737027 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Network error: Client connection negotiation failed: client connection to 127.2.170.194:46217: connect: Connection refused (error 111)
W20251209 03:05:28.737254 8733 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver e40301b8c55143d395ca013de36123cb not available for placement
I20251209 03:05:28.893787 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:05:28.895359 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:28.895444 9104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:28.895360 9106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:28.895530 9103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:28.895673 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:28.895713 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:28.895730 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249528895730 us; error 0 us; skew 500 ppm
I20251209 03:05:28.896355 2731 webserver.cc:492] Webserver started at http://127.2.170.193:40439/ using document root <none> and password file <none>
I20251209 03:05:28.896428 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:28.896462 2731 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251209 03:05:28.898113 2731 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:28.898715 9111 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.898808 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.898857 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "b082d2a61d134d2eb84934cc60a3c82a"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:28.898912 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:28.904024 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:28.904210 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:28.904343 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:28.904613 9118 ts_tablet_manager.cc:542] Loading tablet metadata (0/4 complete)
I20251209 03:05:28.905860 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251209 03:05:28.905907 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:28.905938 2731 ts_tablet_manager.cc:600] Registering tablets (0/4 complete)
I20251209 03:05:28.906369 9118 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap starting.
I20251209 03:05:28.906869 2731 ts_tablet_manager.cc:616] Registered 4 tablets
I20251209 03:05:28.906904 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:28.907328 9118 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:28.907461 9118 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap complete.
I20251209 03:05:28.907593 9118 ts_tablet_manager.cc:1403] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:28.907729 9118 raft_consensus.cc:359] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.907800 9118 raft_consensus.cc:740] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.907850 9118 consensus_queue.cc:260] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.907994 9118 ts_tablet_manager.cc:1434] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.908049 9118 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap starting.
I20251209 03:05:28.908792 9118 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:28.908931 9118 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap complete.
I20251209 03:05:28.909018 9118 ts_tablet_manager.cc:1403] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:28.909126 9118 raft_consensus.cc:359] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:28.909176 9118 raft_consensus.cc:740] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.909219 9118 consensus_queue.cc:260] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:28.909318 9118 ts_tablet_manager.cc:1434] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.909370 9118 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap starting.
I20251209 03:05:28.910099 9118 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:28.910240 9118 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap complete.
I20251209 03:05:28.910334 9118 ts_tablet_manager.cc:1403] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:28.910434 9118 raft_consensus.cc:359] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.910493 9118 raft_consensus.cc:740] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.910538 9118 consensus_queue.cc:260] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.910559 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:45767
I20251209 03:05:28.910601 9179 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:45767 every 8 connection(s)
I20251209 03:05:28.910611 9118 ts_tablet_manager.cc:1434] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:28.910691 9118 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap starting.
I20251209 03:05:28.910840 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:05:28.911829 9118 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
W20251209 03:05:28.911928 9184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:28.912089 9118 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Bootstrap complete.
I20251209 03:05:28.912122 2731 server_base.cc:1047] running on GCE node
I20251209 03:05:28.912194 9118 ts_tablet_manager.cc:1403] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
W20251209 03:05:28.912286 9185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:28.912323 9118 raft_consensus.cc:359] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.912382 9118 raft_consensus.cc:740] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.912442 9118 consensus_queue.cc:260] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.912529 9118 ts_tablet_manager.cc:1434] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20251209 03:05:28.912918 9188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:28.913043 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:28.913075 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:28.913086 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249528913087 us; error 0 us; skew 500 ppm
I20251209 03:05:28.913726 2731 webserver.cc:492] Webserver started at http://127.2.170.194:34049/ using document root <none> and password file <none>
I20251209 03:05:28.913810 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:28.913842 2731 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251209 03:05:28.914461 2731 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20251209 03:05:28.915056 9193 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.915198 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251209 03:05:28.915237 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "e40301b8c55143d395ca013de36123cb"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:28.915287 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:28.917878 9180 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34021
I20251209 03:05:28.917963 9180 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:28.918079 9180 heartbeater.cc:507] Master 127.2.170.254:34021 requested a full tablet report, sending...
I20251209 03:05:28.918426 8683 ts_manager.cc:194] Re-registered known tserver with Master: b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767)
I20251209 03:05:28.918927 8683 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34484
I20251209 03:05:28.919344 9180 heartbeater.cc:499] Master 127.2.170.254:34021 was elected leader, sending a full tablet report...
I20251209 03:05:28.927946 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:28.928135 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:28.928274 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:28.929853 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251209 03:05:28.929886 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:28.930300 9200 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Bootstrap starting.
I20251209 03:05:28.930768 2731 ts_tablet_manager.cc:616] Registered 4 tablets
I20251209 03:05:28.930801 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:28.931368 9200 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251209 03:05:28.931588 9200 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Bootstrap complete.
I20251209 03:05:28.931685 9200 ts_tablet_manager.cc:1403] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:28.931840 9200 raft_consensus.cc:359] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.931921 9200 raft_consensus.cc:740] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40301b8c55143d395ca013de36123cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.932003 9200 consensus_queue.cc:260] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.932164 9200 ts_tablet_manager.cc:1434] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:28.932274 9200 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Bootstrap starting.
I20251209 03:05:28.933579 9200 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251209 03:05:28.933797 9200 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Bootstrap complete.
I20251209 03:05:28.933890 9200 ts_tablet_manager.cc:1403] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:28.933895 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:46217
I20251209 03:05:28.933938 9260 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:46217 every 8 connection(s)
I20251209 03:05:28.934055 9200 raft_consensus.cc:359] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:28.934126 9200 raft_consensus.cc:740] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40301b8c55143d395ca013de36123cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.934170 9200 consensus_queue.cc:260] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:28.934196 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:05:28.934307 9200 ts_tablet_manager.cc:1434] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.934356 9200 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Bootstrap starting.
I20251209 03:05:28.935518 9200 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251209 03:05:28.935698 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:28.935792 9264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:28.935807 9200 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Bootstrap complete.
I20251209 03:05:28.936007 9200 ts_tablet_manager.cc:1403] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251209 03:05:28.936123 9200 raft_consensus.cc:359] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.936170 9200 raft_consensus.cc:740] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40301b8c55143d395ca013de36123cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.936208 9200 consensus_queue.cc:260] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.936334 9200 ts_tablet_manager.cc:1434] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.936396 9200 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Bootstrap starting.
W20251209 03:05:28.936650 9267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:28.936924 9265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:28.937023 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:28.937058 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:28.937075 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249528937075 us; error 0 us; skew 500 ppm
I20251209 03:05:28.937501 9200 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:28.937657 9200 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Bootstrap complete.
I20251209 03:05:28.937777 9200 ts_tablet_manager.cc:1403] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:28.937932 2731 webserver.cc:492] Webserver started at http://127.2.170.195:43751/ using document root <none> and password file <none>
I20251209 03:05:28.937925 9200 raft_consensus.cc:359] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.937988 9200 raft_consensus.cc:740] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40301b8c55143d395ca013de36123cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.938024 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:28.938050 2731 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251209 03:05:28.938033 9200 consensus_queue.cc:260] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.938093 9261 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34021
I20251209 03:05:28.938154 9261 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:28.938117 9200 ts_tablet_manager.cc:1434] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.938234 9261 heartbeater.cc:507] Master 127.2.170.254:34021 requested a full tablet report, sending...
I20251209 03:05:28.938537 8683 ts_manager.cc:194] Re-registered known tserver with Master: e40301b8c55143d395ca013de36123cb (127.2.170.194:46217)
I20251209 03:05:28.938592 2731 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.939023 8683 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34500
I20251209 03:05:28.939327 9274 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:28.939441 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20251209 03:05:28.939494 9261 heartbeater.cc:499] Master 127.2.170.254:34021 was elected leader, sending a full tablet report...
I20251209 03:05:28.939507 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "edb756e223574530abf803c8f40e072b"
format_stamp: "Formatted at 2025-12-09 03:05:21 on dist-test-slave-4cb0"
I20251209 03:05:28.939569 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:28.944008 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:28.944228 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:28.944345 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:28.945632 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (4 total tablets, 4 live tablets)
I20251209 03:05:28.945693 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:28.946101 9281 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Bootstrap starting.
I20251209 03:05:28.946637 2731 ts_tablet_manager.cc:616] Registered 4 tablets
I20251209 03:05:28.946676 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:28.947032 9281 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:28.947175 9281 tablet_bootstrap.cc:492] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Bootstrap complete.
I20251209 03:05:28.947261 9281 ts_tablet_manager.cc:1403] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:28.947410 9281 raft_consensus.cc:359] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.947467 9281 raft_consensus.cc:740] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.947525 9281 consensus_queue.cc:260] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.947681 9281 ts_tablet_manager.cc:1434] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:28.947737 9281 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Bootstrap starting.
I20251209 03:05:28.948889 9281 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:28.949045 9281 tablet_bootstrap.cc:492] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Bootstrap complete.
I20251209 03:05:28.949137 9281 ts_tablet_manager.cc:1403] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:28.949270 9281 raft_consensus.cc:359] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:28.949335 9281 raft_consensus.cc:740] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.949378 9281 consensus_queue.cc:260] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:28.949468 9281 ts_tablet_manager.cc:1434] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:28.949518 9281 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Bootstrap starting.
I20251209 03:05:28.950099 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:34053
I20251209 03:05:28.950395 9341 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:34053 every 8 connection(s)
I20251209 03:05:28.950580 9281 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:28.950729 9281 tablet_bootstrap.cc:492] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Bootstrap complete.
I20251209 03:05:28.950840 9281 ts_tablet_manager.cc:1403] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:28.950994 9281 raft_consensus.cc:359] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.951063 9281 raft_consensus.cc:740] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.951115 9281 consensus_queue.cc:260] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.951200 9281 ts_tablet_manager.cc:1434] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:28.951254 9281 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Bootstrap starting.
I20251209 03:05:28.952119 9281 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:28.952275 9281 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Bootstrap complete.
I20251209 03:05:28.952370 9281 ts_tablet_manager.cc:1403] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:28.952512 9281 raft_consensus.cc:359] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.952585 9281 raft_consensus.cc:740] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Initialized, Role: FOLLOWER
I20251209 03:05:28.952687 9281 consensus_queue.cc:260] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:28.952800 9281 ts_tablet_manager.cc:1434] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:28.953644 9342 heartbeater.cc:344] Connected to a master server at 127.2.170.254:34021
I20251209 03:05:28.953766 9342 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:28.953877 9342 heartbeater.cc:507] Master 127.2.170.254:34021 requested a full tablet report, sending...
I20251209 03:05:28.954149 8683 ts_manager.cc:194] Re-registered known tserver with Master: edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:28.954521 8683 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34504
I20251209 03:05:28.954871 9342 heartbeater.cc:499] Master 127.2.170.254:34021 was elected leader, sending a full tablet report...
W20251209 03:05:29.740694 8733 auto_rebalancer.cc:253] failed to send replica move request: Illegal state: Replica e40301b8c55143d395ca013de36123cb is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } }
W20251209 03:05:29.741120 8733 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 77cdfff2712448f280f34a91ead8a040, TS b082d2a61d134d2eb84934cc60a3c82a -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
W20251209 03:05:29.741165 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 77cdfff2712448f280f34a91ead8a040, TS b082d2a61d134d2eb84934cc60a3c82a -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
W20251209 03:05:29.741453 8733 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 905884ffefc9486c8fc0a8fac0fb6b98, TS edb756e223574530abf803c8f40e072b -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
W20251209 03:05:29.741487 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 905884ffefc9486c8fc0a8fac0fb6b98, TS edb756e223574530abf803c8f40e072b -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
W20251209 03:05:29.744076 8733 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 01d23b8eaeea4bc983d0c12df54ff83f, TS e40301b8c55143d395ca013de36123cb -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
W20251209 03:05:29.744122 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 01d23b8eaeea4bc983d0c12df54ff83f, TS e40301b8c55143d395ca013de36123cb -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.040585 9350 raft_consensus.cc:493] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:30.040686 9350 raft_consensus.cc:515] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.040889 9350 leader_election.cc:290] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e40301b8c55143d395ca013de36123cb (127.2.170.194:46217), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.044029 9234 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d23b8eaeea4bc983d0c12df54ff83f" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb" is_pre_election: true
I20251209 03:05:30.044121 9234 raft_consensus.cc:2468] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 1.
I20251209 03:05:30.044205 9316 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d23b8eaeea4bc983d0c12df54ff83f" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:30.044255 9316 raft_consensus.cc:2468] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 1.
I20251209 03:05:30.044428 9113 leader_election.cc:304] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:30.044535 9350 raft_consensus.cc:2804] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:05:30.044579 9350 raft_consensus.cc:493] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:30.044600 9350 raft_consensus.cc:3060] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.045189 9350 raft_consensus.cc:515] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.045303 9350 leader_election.cc:290] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 election: Requested vote from peers e40301b8c55143d395ca013de36123cb (127.2.170.194:46217), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.045473 9316 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d23b8eaeea4bc983d0c12df54ff83f" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:30.045454 9234 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d23b8eaeea4bc983d0c12df54ff83f" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb"
I20251209 03:05:30.045544 9316 raft_consensus.cc:3060] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.045552 9234 raft_consensus.cc:3060] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.046255 9234 raft_consensus.cc:2468] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 2.
I20251209 03:05:30.046255 9316 raft_consensus.cc:2468] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 2.
I20251209 03:05:30.046460 9113 leader_election.cc:304] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, e40301b8c55143d395ca013de36123cb; no voters:
I20251209 03:05:30.046554 9350 raft_consensus.cc:2804] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:05:30.046680 9350 raft_consensus.cc:697] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Becoming Leader. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Running, Role: LEADER
I20251209 03:05:30.046754 9350 consensus_queue.cc:237] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.047330 8683 catalog_manager.cc:5654] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "b082d2a61d134d2eb84934cc60a3c82a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:30.184821 9360 raft_consensus.cc:493] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:30.184931 9360 raft_consensus.cc:515] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.185123 9360 leader_election.cc:290] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), e40301b8c55143d395ca013de36123cb (127.2.170.194:46217)
I20251209 03:05:30.187809 9234 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "edb756e223574530abf803c8f40e072b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb" is_pre_election: true
I20251209 03:05:30.187868 9146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "edb756e223574530abf803c8f40e072b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a" is_pre_election: true
I20251209 03:05:30.187930 9234 raft_consensus.cc:2410] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate edb756e223574530abf803c8f40e072b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251209 03:05:30.187940 9146 raft_consensus.cc:2468] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edb756e223574530abf803c8f40e072b in term 1.
I20251209 03:05:30.188198 9276 leader_election.cc:304] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, edb756e223574530abf803c8f40e072b; no voters: e40301b8c55143d395ca013de36123cb
I20251209 03:05:30.188292 9360 raft_consensus.cc:2804] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:05:30.188333 9360 raft_consensus.cc:493] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:30.188346 9360 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.188941 9360 raft_consensus.cc:515] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.189040 9360 leader_election.cc:290] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [CANDIDATE]: Term 2 election: Requested vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), e40301b8c55143d395ca013de36123cb (127.2.170.194:46217)
I20251209 03:05:30.189173 9146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "edb756e223574530abf803c8f40e072b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a"
I20251209 03:05:30.189225 9234 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "edb756e223574530abf803c8f40e072b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb"
I20251209 03:05:30.189229 9146 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.189306 9234 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.189926 9234 raft_consensus.cc:2410] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate edb756e223574530abf803c8f40e072b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251209 03:05:30.190030 9146 raft_consensus.cc:2468] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edb756e223574530abf803c8f40e072b in term 2.
I20251209 03:05:30.190167 9276 leader_election.cc:304] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, edb756e223574530abf803c8f40e072b; no voters: e40301b8c55143d395ca013de36123cb
I20251209 03:05:30.190250 9360 raft_consensus.cc:2804] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:05:30.190398 9360 raft_consensus.cc:697] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 2 LEADER]: Becoming Leader. State: Replica: edb756e223574530abf803c8f40e072b, State: Running, Role: LEADER
I20251209 03:05:30.190452 9360 consensus_queue.cc:237] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.190951 8683 catalog_manager.cc:5654] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b reported cstate change: term changed from 1 to 2, leader changed from e40301b8c55143d395ca013de36123cb (127.2.170.194) to edb756e223574530abf803c8f40e072b (127.2.170.195). New cstate: current_term: 2 leader_uuid: "edb756e223574530abf803c8f40e072b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: HEALTHY } } }
I20251209 03:05:30.254890 9350 raft_consensus.cc:493] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:30.254997 9350 raft_consensus.cc:515] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:30.255152 9350 leader_election.cc:290] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e40301b8c55143d395ca013de36123cb (127.2.170.194:46217), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.255391 9234 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb" is_pre_election: true
I20251209 03:05:30.255391 9316 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:30.255525 9234 raft_consensus.cc:2410] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b082d2a61d134d2eb84934cc60a3c82a for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251209 03:05:30.255527 9316 raft_consensus.cc:2468] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 1.
I20251209 03:05:30.255722 9113 leader_election.cc:304] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, edb756e223574530abf803c8f40e072b; no voters: e40301b8c55143d395ca013de36123cb
I20251209 03:05:30.255805 9350 raft_consensus.cc:2804] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:05:30.255854 9350 raft_consensus.cc:493] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:30.255882 9350 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.256573 9350 raft_consensus.cc:515] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:30.256685 9350 leader_election.cc:290] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 election: Requested vote from peers e40301b8c55143d395ca013de36123cb (127.2.170.194:46217), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.256821 9234 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb"
I20251209 03:05:30.256879 9234 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.256910 9316 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:30.256978 9316 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.257444 9234 raft_consensus.cc:2410] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b082d2a61d134d2eb84934cc60a3c82a for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251209 03:05:30.257563 9316 raft_consensus.cc:2468] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 2.
I20251209 03:05:30.257714 9113 leader_election.cc:304] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, edb756e223574530abf803c8f40e072b; no voters: e40301b8c55143d395ca013de36123cb
I20251209 03:05:30.257797 9350 raft_consensus.cc:2804] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:05:30.257851 9350 raft_consensus.cc:697] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Becoming Leader. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Running, Role: LEADER
I20251209 03:05:30.257901 9350 consensus_queue.cc:237] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:30.258499 8683 catalog_manager.cc:5654] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a reported cstate change: term changed from 1 to 2, leader changed from e40301b8c55143d395ca013de36123cb (127.2.170.194) to b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193). New cstate: current_term: 2 leader_uuid: "b082d2a61d134d2eb84934cc60a3c82a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: HEALTHY } } }
I20251209 03:05:30.473708 9350 raft_consensus.cc:493] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:30.473806 9350 raft_consensus.cc:515] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.473948 9350 leader_election.cc:290] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e40301b8c55143d395ca013de36123cb (127.2.170.194:46217), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.474066 9234 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251209 03:05:30.474131 9316 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:30.474196 9316 raft_consensus.cc:2468] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 1.
I20251209 03:05:30.474134 9233 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb" is_pre_election: true
I20251209 03:05:30.474287 9233 raft_consensus.cc:2410] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b082d2a61d134d2eb84934cc60a3c82a for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251209 03:05:30.474223 9350 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:30.474382 9113 leader_election.cc:304] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:30.474471 9354 raft_consensus.cc:2804] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:05:30.474522 9354 raft_consensus.cc:493] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:30.474540 9354 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.475399 9354 raft_consensus.cc:515] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.475508 9354 leader_election.cc:290] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 election: Requested vote from peers e40301b8c55143d395ca013de36123cb (127.2.170.194:46217), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.475634 9234 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e40301b8c55143d395ca013de36123cb"
I20251209 03:05:30.475687 9234 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.475840 9316 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "b082d2a61d134d2eb84934cc60a3c82a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:30.475886 9316 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:30.476066 9315 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251209 03:05:30.476235 9368 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:30.476411 9316 raft_consensus.cc:2468] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b082d2a61d134d2eb84934cc60a3c82a in term 2.
I20251209 03:05:30.476668 9113 leader_election.cc:304] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:30.476778 9354 raft_consensus.cc:2804] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:05:30.476889 9354 raft_consensus.cc:697] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Becoming Leader. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Running, Role: LEADER
I20251209 03:05:30.476938 9354 consensus_queue.cc:237] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.476411 9234 raft_consensus.cc:2410] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b082d2a61d134d2eb84934cc60a3c82a for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20251209 03:05:30.477373 8683 catalog_manager.cc:5654] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a reported cstate change: term changed from 1 to 2, leader changed from e40301b8c55143d395ca013de36123cb (127.2.170.194) to b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193). New cstate: current_term: 2 leader_uuid: "b082d2a61d134d2eb84934cc60a3c82a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:30.482787 9372 raft_consensus.cc:493] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:30.482848 9372 raft_consensus.cc:515] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.482980 9372 leader_election.cc:290] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.485888 9315 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:30.485937 9146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a" is_pre_election: true
I20251209 03:05:30.485996 9146 raft_consensus.cc:2468] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 2.
I20251209 03:05:30.486153 9195 leader_election.cc:304] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, e40301b8c55143d395ca013de36123cb; no voters: edb756e223574530abf803c8f40e072b
I20251209 03:05:30.486240 9372 raft_consensus.cc:2804] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251209 03:05:30.486287 9372 raft_consensus.cc:493] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:30.486312 9372 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:30.486884 9372 raft_consensus.cc:515] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.486989 9372 leader_election.cc:290] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 election: Requested vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.487119 9146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a"
I20251209 03:05:30.487125 9315 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f662583a5c641ab946b9fb216fea5fa" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:30.487182 9146 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:30.487699 9146 raft_consensus.cc:2468] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 3.
I20251209 03:05:30.487833 9195 leader_election.cc:304] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b082d2a61d134d2eb84934cc60a3c82a, e40301b8c55143d395ca013de36123cb; no voters: edb756e223574530abf803c8f40e072b
I20251209 03:05:30.487908 9372 raft_consensus.cc:2804] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:05:30.487953 9372 raft_consensus.cc:697] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 3 LEADER]: Becoming Leader. State: Replica: e40301b8c55143d395ca013de36123cb, State: Running, Role: LEADER
I20251209 03:05:30.487989 9372 consensus_queue.cc:237] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.488474 8683 catalog_manager.cc:5654] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb reported cstate change: term changed from 2 to 3, leader changed from edb756e223574530abf803c8f40e072b (127.2.170.195) to e40301b8c55143d395ca013de36123cb (127.2.170.194). New cstate: current_term: 3 leader_uuid: "e40301b8c55143d395ca013de36123cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:30.613714 9372 raft_consensus.cc:493] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:30.613823 9372 raft_consensus.cc:515] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.613965 9372 leader_election.cc:290] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.614220 9315 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:30.614194 9146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a" is_pre_election: true
I20251209 03:05:30.614280 9315 raft_consensus.cc:2468] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 2.
I20251209 03:05:30.614396 9195 leader_election.cc:304] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters:
I20251209 03:05:30.614470 9372 raft_consensus.cc:2804] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251209 03:05:30.614508 9372 raft_consensus.cc:493] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:30.614530 9372 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:30.615145 9372 raft_consensus.cc:515] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.615263 9372 leader_election.cc:290] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 election: Requested vote from peers b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767), edb756e223574530abf803c8f40e072b (127.2.170.195:34053)
I20251209 03:05:30.615435 9146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a"
I20251209 03:05:30.615473 9315 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "905884ffefc9486c8fc0a8fac0fb6b98" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:30.615549 9315 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:30.616159 9315 raft_consensus.cc:2468] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 3.
I20251209 03:05:30.616321 9195 leader_election.cc:304] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters: b082d2a61d134d2eb84934cc60a3c82a
I20251209 03:05:30.616415 9372 raft_consensus.cc:2804] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:05:30.616477 9372 raft_consensus.cc:697] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 3 LEADER]: Becoming Leader. State: Replica: e40301b8c55143d395ca013de36123cb, State: Running, Role: LEADER
I20251209 03:05:30.616516 9372 consensus_queue.cc:237] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.617028 8683 catalog_manager.cc:5654] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb reported cstate change: term changed from 2 to 3, leader changed from b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193) to e40301b8c55143d395ca013de36123cb (127.2.170.194). New cstate: current_term: 3 leader_uuid: "e40301b8c55143d395ca013de36123cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:30.660902 9146 raft_consensus.cc:1240] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Rejecting Update request from peer edb756e223574530abf803c8f40e072b for earlier term 2. Current term is 3. Ops: []
I20251209 03:05:30.661198 9363 consensus_queue.cc:1059] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:30.661340 9363 raft_consensus.cc:3055] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 2 LEADER]: Stepping down as leader of term 2
I20251209 03:05:30.661373 9363 raft_consensus.cc:740] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: edb756e223574530abf803c8f40e072b, State: Running, Role: LEADER
I20251209 03:05:30.661428 9363 consensus_queue.cc:260] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.661499 9363 raft_consensus.cc:3060] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:30.705790 9372 raft_consensus.cc:493] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:30.705888 9372 raft_consensus.cc:515] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:30.706043 9372 leader_election.cc:290] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edb756e223574530abf803c8f40e072b (127.2.170.195:34053), b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767)
I20251209 03:05:30.706241 9315 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b" is_pre_election: true
I20251209 03:05:30.706247 9146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a" is_pre_election: true
I20251209 03:05:30.706357 9315 raft_consensus.cc:2468] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 2.
I20251209 03:05:30.706512 9195 leader_election.cc:304] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters: b082d2a61d134d2eb84934cc60a3c82a
I20251209 03:05:30.706597 9372 raft_consensus.cc:2804] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251209 03:05:30.706640 9372 raft_consensus.cc:493] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:30.706658 9372 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:30.707219 9372 raft_consensus.cc:515] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:30.707317 9372 leader_election.cc:290] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 election: Requested vote from peers edb756e223574530abf803c8f40e072b (127.2.170.195:34053), b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767)
I20251209 03:05:30.707485 9146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b082d2a61d134d2eb84934cc60a3c82a"
I20251209 03:05:30.707495 9315 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "77cdfff2712448f280f34a91ead8a040" candidate_uuid: "e40301b8c55143d395ca013de36123cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "edb756e223574530abf803c8f40e072b"
I20251209 03:05:30.707595 9315 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:30.708143 9315 raft_consensus.cc:2468] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40301b8c55143d395ca013de36123cb in term 3.
I20251209 03:05:30.708293 9195 leader_election.cc:304] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e40301b8c55143d395ca013de36123cb, edb756e223574530abf803c8f40e072b; no voters: b082d2a61d134d2eb84934cc60a3c82a
I20251209 03:05:30.708374 9372 raft_consensus.cc:2804] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Leader election won for term 3
I20251209 03:05:30.708416 9372 raft_consensus.cc:697] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 3 LEADER]: Becoming Leader. State: Replica: e40301b8c55143d395ca013de36123cb, State: Running, Role: LEADER
I20251209 03:05:30.708459 9372 consensus_queue.cc:237] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:30.709091 8683 catalog_manager.cc:5654] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb reported cstate change: term changed from 2 to 3, leader changed from b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193) to e40301b8c55143d395ca013de36123cb (127.2.170.194). New cstate: current_term: 3 leader_uuid: "e40301b8c55143d395ca013de36123cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:30.744916 9146 consensus_queue.cc:237] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: true } }
I20251209 03:05:30.745438 9315 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251209 03:05:30.745438 9234 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251209 03:05:30.745703 9354 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:30.745836 9354 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:30.746274 9368 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c70df044711546178be1fc7bf0f77dfd (127.2.170.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: true } } }
I20251209 03:05:30.746517 9234 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c70df044711546178be1fc7bf0f77dfd (127.2.170.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: true } } }
I20251209 03:05:30.746639 9315 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c70df044711546178be1fc7bf0f77dfd (127.2.170.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: true } } }
I20251209 03:05:30.747561 8683 catalog_manager.cc:5654] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a reported cstate change: config changed from index -1 to 3, NON_VOTER c70df044711546178be1fc7bf0f77dfd (127.2.170.196) added. New cstate: current_term: 2 leader_uuid: "b082d2a61d134d2eb84934cc60a3c82a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251209 03:05:30.747987 8733 auto_rebalancer.cc:253] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
W20251209 03:05:30.748844 8733 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 905884ffefc9486c8fc0a8fac0fb6b98, TS b082d2a61d134d2eb84934cc60a3c82a -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
W20251209 03:05:30.748893 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 905884ffefc9486c8fc0a8fac0fb6b98, TS b082d2a61d134d2eb84934cc60a3c82a -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
W20251209 03:05:30.749361 8733 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 2f662583a5c641ab946b9fb216fea5fa, TS edb756e223574530abf803c8f40e072b -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
W20251209 03:05:30.749400 8733 auto_rebalancer.cc:263] scheduled replica move failed to complete: Incomplete: tablet 2f662583a5c641ab946b9fb216fea5fa, TS edb756e223574530abf803c8f40e072b -> TS c70df044711546178be1fc7bf0f77dfd move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
W20251209 03:05:30.750279 9115 consensus_peers.cc:597] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a -> Peer c70df044711546178be1fc7bf0f77dfd (127.2.170.196:41917): Couldn't send request to peer c70df044711546178be1fc7bf0f77dfd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01d23b8eaeea4bc983d0c12df54ff83f. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:30.859884 9315 raft_consensus.cc:1240] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Rejecting Update request from peer b082d2a61d134d2eb84934cc60a3c82a for earlier term 2. Current term is 3. Ops: []
I20251209 03:05:30.860356 9354 consensus_queue.cc:1059] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:30.860518 9354 raft_consensus.cc:3055] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Stepping down as leader of term 2
I20251209 03:05:30.860558 9354 raft_consensus.cc:740] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Running, Role: LEADER
I20251209 03:05:30.860611 9354 consensus_queue.cc:260] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } }
I20251209 03:05:30.860836 9354 raft_consensus.cc:3060] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:30.899184 9146 raft_consensus.cc:1275] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Refusing update from remote peer e40301b8c55143d395ca013de36123cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251209 03:05:30.899446 9372 consensus_queue.cc:1048] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:30.900720 9315 raft_consensus.cc:1275] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Refusing update from remote peer e40301b8c55143d395ca013de36123cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251209 03:05:30.900897 9372 consensus_queue.cc:1048] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:30.900977 9372 consensus_queue.cc:1243] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [LEADER]: Peer edb756e223574530abf803c8f40e072b log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251209 03:05:30.901227 9315 pending_rounds.cc:85] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Aborting all ops after (but not including) 0
I20251209 03:05:30.901263 9315 pending_rounds.cc:107] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251209 03:05:30.901281 9315 raft_consensus.cc:2889] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251209 03:05:30.901997 9315 raft_consensus.cc:1240] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Rejecting Update request from peer b082d2a61d134d2eb84934cc60a3c82a for earlier term 2. Current term is 3. Ops: []
I20251209 03:05:30.902281 9354 consensus_queue.cc:1059] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:30.902377 9354 raft_consensus.cc:3055] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Stepping down as leader of term 2
I20251209 03:05:30.902408 9354 raft_consensus.cc:740] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b082d2a61d134d2eb84934cc60a3c82a, State: Running, Role: LEADER
I20251209 03:05:30.902453 9354 consensus_queue.cc:260] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } }
I20251209 03:05:30.902526 9354 raft_consensus.cc:3060] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Advancing to term 3
I20251209 03:05:31.033531 9146 raft_consensus.cc:1275] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Refusing update from remote peer e40301b8c55143d395ca013de36123cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251209 03:05:31.033900 9372 consensus_queue.cc:1048] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:31.034034 9372 consensus_queue.cc:1243] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [LEADER]: Peer b082d2a61d134d2eb84934cc60a3c82a log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251209 03:05:31.034330 9146 pending_rounds.cc:85] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Aborting all ops after (but not including) 0
I20251209 03:05:31.034384 9146 pending_rounds.cc:107] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251209 03:05:31.034403 9146 raft_consensus.cc:2889] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251209 03:05:31.035293 9315 raft_consensus.cc:1275] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Refusing update from remote peer e40301b8c55143d395ca013de36123cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251209 03:05:31.035537 9384 consensus_queue.cc:1048] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:31.098243 9315 raft_consensus.cc:1275] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Refusing update from remote peer e40301b8c55143d395ca013de36123cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251209 03:05:31.098474 9384 consensus_queue.cc:1048] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:31.099740 9146 raft_consensus.cc:1275] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Refusing update from remote peer e40301b8c55143d395ca013de36123cb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20251209 03:05:31.100247 9384 consensus_queue.cc:1048] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:31.100337 9384 consensus_queue.cc:1243] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [LEADER]: Peer b082d2a61d134d2eb84934cc60a3c82a log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20251209 03:05:31.100553 9153 pending_rounds.cc:85] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Aborting all ops after (but not including) 0
I20251209 03:05:31.100603 9153 pending_rounds.cc:107] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20251209 03:05:31.100622 9153 raft_consensus.cc:2889] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251209 03:05:31.229766 9387 ts_tablet_manager.cc:933] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: Initiating tablet copy from peer b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767)
I20251209 03:05:31.229974 9387 tablet_copy_client.cc:323] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.193:45767
I20251209 03:05:31.233445 9163 tablet_copy_service.cc:140] P b082d2a61d134d2eb84934cc60a3c82a: Received BeginTabletCopySession request for tablet 01d23b8eaeea4bc983d0c12df54ff83f from peer c70df044711546178be1fc7bf0f77dfd ({username='slave'} at 127.0.0.1:40664)
I20251209 03:05:31.233522 9163 tablet_copy_service.cc:161] P b082d2a61d134d2eb84934cc60a3c82a: Beginning new tablet copy session on tablet 01d23b8eaeea4bc983d0c12df54ff83f from peer c70df044711546178be1fc7bf0f77dfd at {username='slave'} at 127.0.0.1:40664: session id = c70df044711546178be1fc7bf0f77dfd-01d23b8eaeea4bc983d0c12df54ff83f
I20251209 03:05:31.234141 9163 tablet_copy_source_session.cc:215] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:05:31.234467 9387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d23b8eaeea4bc983d0c12df54ff83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:31.235421 9387 tablet_copy_client.cc:806] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: tablet copy: Starting download of 0 data blocks...
I20251209 03:05:31.235562 9387 tablet_copy_client.cc:670] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: tablet copy: Starting download of 1 WAL segments...
I20251209 03:05:31.236060 9387 tablet_copy_client.cc:538] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:05:31.236654 9387 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: Bootstrap starting.
I20251209 03:05:31.238011 9387 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:31.238180 9387 tablet_bootstrap.cc:492] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: Bootstrap complete.
I20251209 03:05:31.238270 9387 ts_tablet_manager.cc:1403] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:31.238432 9387 raft_consensus.cc:359] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: true } }
I20251209 03:05:31.238498 9387 raft_consensus.cc:740] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c70df044711546178be1fc7bf0f77dfd, State: Initialized, Role: LEARNER
I20251209 03:05:31.238528 9387 consensus_queue.cc:260] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: true } }
I20251209 03:05:31.238693 9387 ts_tablet_manager.cc:1434] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20251209 03:05:31.238878 9163 tablet_copy_service.cc:342] P b082d2a61d134d2eb84934cc60a3c82a: Request end of tablet copy session c70df044711546178be1fc7bf0f77dfd-01d23b8eaeea4bc983d0c12df54ff83f received from {username='slave'} at 127.0.0.1:40664
I20251209 03:05:31.238920 9163 tablet_copy_service.cc:434] P b082d2a61d134d2eb84934cc60a3c82a: ending tablet copy session c70df044711546178be1fc7bf0f77dfd-01d23b8eaeea4bc983d0c12df54ff83f on tablet 01d23b8eaeea4bc983d0c12df54ff83f with peer c70df044711546178be1fc7bf0f77dfd
I20251209 03:05:31.793798 9058 raft_consensus.cc:1217] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20251209 03:05:32.390239 9354 raft_consensus.cc:1064] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a: attempting to promote NON_VOTER c70df044711546178be1fc7bf0f77dfd to VOTER
I20251209 03:05:32.390446 9354 consensus_queue.cc:237] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } }
I20251209 03:05:32.390923 9315 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251209 03:05:32.390942 9234 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251209 03:05:32.391060 9058 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 LEARNER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251209 03:05:32.391126 9397 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:32.391294 9354 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:32.391433 9398 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:32.392150 9354 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c70df044711546178be1fc7bf0f77dfd (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.392537 9315 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c70df044711546178be1fc7bf0f77dfd (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.393364 9058 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c70df044711546178be1fc7bf0f77dfd (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.393633 9234 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c70df044711546178be1fc7bf0f77dfd (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.394481 8684 catalog_manager.cc:5654] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b reported cstate change: config changed from index 3 to 4, c70df044711546178be1fc7bf0f77dfd (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "b082d2a61d134d2eb84934cc60a3c82a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "e40301b8c55143d395ca013de36123cb" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 46217 } attrs { replace: true } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.395848 9146 consensus_queue.cc:237] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } }
I20251209 03:05:32.396162 9058 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 FOLLOWER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251209 03:05:32.396229 9315 raft_consensus.cc:1275] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Refusing update from remote peer b082d2a61d134d2eb84934cc60a3c82a: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20251209 03:05:32.396363 9398 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:32.396533 9398 consensus_queue.cc:1048] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:32.396875 9398 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e40301b8c55143d395ca013de36123cb (127.2.170.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.396968 9058 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e40301b8c55143d395ca013de36123cb (127.2.170.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.397065 9315 raft_consensus.cc:2955] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e40301b8c55143d395ca013de36123cb (127.2.170.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.397881 8728 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 01d23b8eaeea4bc983d0c12df54ff83f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:32.397881 8681 catalog_manager.cc:5654] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b reported cstate change: config changed from index 4 to 5, VOTER e40301b8c55143d395ca013de36123cb (127.2.170.194) evicted. New cstate: current_term: 2 leader_uuid: "b082d2a61d134d2eb84934cc60a3c82a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b082d2a61d134d2eb84934cc60a3c82a" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 45767 } } peers { permanent_uuid: "edb756e223574530abf803c8f40e072b" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 34053 } } peers { permanent_uuid: "c70df044711546178be1fc7bf0f77dfd" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 41917 } attrs { promote: false } } }
I20251209 03:05:32.400949 9223 tablet_service.cc:1552] Processing DeleteTablet for tablet 01d23b8eaeea4bc983d0c12df54ff83f with delete_type TABLET_DATA_TOMBSTONED (TS e40301b8c55143d395ca013de36123cb not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41110
I20251209 03:05:32.401166 9410 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:32.401229 9410 raft_consensus.cc:2243] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:32.401274 9410 raft_consensus.cc:2272] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:32.401510 9410 ts_tablet_manager.cc:1916] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:05:32.402876 9410 ts_tablet_manager.cc:1929] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20251209 03:05:32.402923 9410 log.cc:1199] T 01d23b8eaeea4bc983d0c12df54ff83f P e40301b8c55143d395ca013de36123cb: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestHandlingFailedTservers.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/01d23b8eaeea4bc983d0c12df54ff83f
I20251209 03:05:32.403239 8669 catalog_manager.cc:4985] TS e40301b8c55143d395ca013de36123cb (127.2.170.194:46217): tablet 01d23b8eaeea4bc983d0c12df54ff83f (table test-workload [id=05b20a28b651405ab3876d42405b45fe]) successfully deleted
I20251209 03:05:32.975347 2731 tablet_server.cc:178] TabletServer@127.2.170.193:45767 shutting down...
W20251209 03:05:32.976852 9195 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20251209 03:05:32.977775 9195 consensus_peers.cc:597] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb -> Peer b082d2a61d134d2eb84934cc60a3c82a (127.2.170.193:45767): Couldn't send request to peer b082d2a61d134d2eb84934cc60a3c82a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:32.978466 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:32.978571 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:32.978628 2731 raft_consensus.cc:2243] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:32.978677 2731 raft_consensus.cc:2272] T 905884ffefc9486c8fc0a8fac0fb6b98 P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:32.978832 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:32.978868 2731 raft_consensus.cc:2243] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:32.978896 2731 raft_consensus.cc:2272] T 2f662583a5c641ab946b9fb216fea5fa P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:32.979022 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:32.979053 2731 raft_consensus.cc:2243] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:05:32.979111 2731 raft_consensus.cc:2272] T 01d23b8eaeea4bc983d0c12df54ff83f P b082d2a61d134d2eb84934cc60a3c82a [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:32.979482 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:32.979545 2731 raft_consensus.cc:2243] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:32.979583 2731 raft_consensus.cc:2272] T 77cdfff2712448f280f34a91ead8a040 P b082d2a61d134d2eb84934cc60a3c82a [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:32.991703 2731 tablet_server.cc:195] TabletServer@127.2.170.193:45767 shutdown complete.
I20251209 03:05:32.992822 2731 tablet_server.cc:178] TabletServer@127.2.170.194:46217 shutting down...
I20251209 03:05:32.995361 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:32.995500 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:32.995559 2731 raft_consensus.cc:2243] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 3 LEADER]: Raft consensus shutting down.
I20251209 03:05:32.995630 2731 raft_consensus.cc:2272] T 905884ffefc9486c8fc0a8fac0fb6b98 P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:32.995936 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:32.995991 2731 raft_consensus.cc:2243] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 3 LEADER]: Raft consensus shutting down.
I20251209 03:05:32.996064 2731 raft_consensus.cc:2272] T 2f662583a5c641ab946b9fb216fea5fa P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:32.996219 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:32.996253 2731 raft_consensus.cc:2243] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 3 LEADER]: Raft consensus shutting down.
I20251209 03:05:32.996299 2731 raft_consensus.cc:2272] T 77cdfff2712448f280f34a91ead8a040 P e40301b8c55143d395ca013de36123cb [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:33.008538 2731 tablet_server.cc:195] TabletServer@127.2.170.194:46217 shutdown complete.
I20251209 03:05:33.009661 2731 tablet_server.cc:178] TabletServer@127.2.170.195:34053 shutting down...
I20251209 03:05:33.012563 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:33.012662 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:33.012720 2731 raft_consensus.cc:2243] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:33.012768 2731 raft_consensus.cc:2272] T 905884ffefc9486c8fc0a8fac0fb6b98 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:33.012926 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:33.012964 2731 raft_consensus.cc:2243] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:33.012991 2731 raft_consensus.cc:2272] T 2f662583a5c641ab946b9fb216fea5fa P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:33.013096 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:33.013127 2731 raft_consensus.cc:2243] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:33.013172 2731 raft_consensus.cc:2272] T 01d23b8eaeea4bc983d0c12df54ff83f P edb756e223574530abf803c8f40e072b [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:33.013348 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:33.013383 2731 raft_consensus.cc:2243] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:33.013410 2731 raft_consensus.cc:2272] T 77cdfff2712448f280f34a91ead8a040 P edb756e223574530abf803c8f40e072b [term 3 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:33.024932 2731 tablet_server.cc:195] TabletServer@127.2.170.195:34053 shutdown complete.
I20251209 03:05:33.026029 2731 tablet_server.cc:178] TabletServer@127.2.170.196:0 shutting down...
I20251209 03:05:33.028533 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:33.028643 2731 tablet_replica.cc:333] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd: stopping tablet replica
I20251209 03:05:33.028692 2731 raft_consensus.cc:2243] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:33.028753 2731 raft_consensus.cc:2272] T 01d23b8eaeea4bc983d0c12df54ff83f P c70df044711546178be1fc7bf0f77dfd [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:33.040163 2731 tablet_server.cc:195] TabletServer@127.2.170.196:0 shutdown complete.
I20251209 03:05:33.041033 2731 master.cc:561] Master@127.2.170.254:34021 shutting down...
I20251209 03:05:33.044384 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:33.044519 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:33.044564 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P c18f8c6c2b864102a93a5e7692650c5e: stopping tablet replica
I20251209 03:05:33.056504 2731 master.cc:583] Master@127.2.170.254:34021 shutdown complete.
[ OK ] AutoRebalancerTest.TestHandlingFailedTservers (11364 ms)
[ RUN ] AutoRebalancerTest.TestDeletedTables
I20251209 03:05:33.062258 2731 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.2.170.254:41961
I20251209 03:05:33.062407 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:33.063604 9413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:33.063637 9412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:33.063791 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:33.063706 9415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:33.064052 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:33.064090 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:33.064101 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249533064101 us; error 0 us; skew 500 ppm
I20251209 03:05:33.064626 2731 webserver.cc:492] Webserver started at http://127.2.170.254:42353/ using document root <none> and password file <none>
I20251209 03:05:33.064694 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:33.064738 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:33.064780 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:33.065019 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/master-0-root/instance:
uuid: "e81c8b1e88c549b18ba480b79eba8d47"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:33.065896 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:33.066319 9420 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.066429 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.066473 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/master-0-root
uuid: "e81c8b1e88c549b18ba480b79eba8d47"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:33.066516 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:33.081050 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:33.081254 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:33.083859 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.254:41961
I20251209 03:05:33.086222 9472 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.254:41961 every 8 connection(s)
I20251209 03:05:33.086336 9473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:33.087342 9473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: Bootstrap starting.
I20251209 03:05:33.087657 9473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:33.088116 9473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: No bootstrap required, opened a new log
I20251209 03:05:33.088230 9473 raft_consensus.cc:359] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c8b1e88c549b18ba480b79eba8d47" member_type: VOTER }
I20251209 03:05:33.088284 9473 raft_consensus.cc:385] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:33.088297 9473 raft_consensus.cc:740] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e81c8b1e88c549b18ba480b79eba8d47, State: Initialized, Role: FOLLOWER
I20251209 03:05:33.088337 9473 consensus_queue.cc:260] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c8b1e88c549b18ba480b79eba8d47" member_type: VOTER }
I20251209 03:05:33.088372 9473 raft_consensus.cc:399] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251209 03:05:33.088397 9473 raft_consensus.cc:493] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251209 03:05:33.088421 9473 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:33.089154 9473 raft_consensus.cc:515] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c8b1e88c549b18ba480b79eba8d47" member_type: VOTER }
I20251209 03:05:33.089210 9473 leader_election.cc:304] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e81c8b1e88c549b18ba480b79eba8d47; no voters:
I20251209 03:05:33.089296 9473 leader_election.cc:290] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251209 03:05:33.089381 9476 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:33.089426 9473 sys_catalog.cc:565] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [sys.catalog]: configured and running, proceeding with master startup.
I20251209 03:05:33.089529 9476 raft_consensus.cc:697] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 1 LEADER]: Becoming Leader. State: Replica: e81c8b1e88c549b18ba480b79eba8d47, State: Running, Role: LEADER
I20251209 03:05:33.089582 9476 consensus_queue.cc:237] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c8b1e88c549b18ba480b79eba8d47" member_type: VOTER }
I20251209 03:05:33.089811 9477 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e81c8b1e88c549b18ba480b79eba8d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c8b1e88c549b18ba480b79eba8d47" member_type: VOTER } }
I20251209 03:05:33.089859 9478 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e81c8b1e88c549b18ba480b79eba8d47. Latest consensus state: current_term: 1 leader_uuid: "e81c8b1e88c549b18ba480b79eba8d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81c8b1e88c549b18ba480b79eba8d47" member_type: VOTER } }
I20251209 03:05:33.089938 9478 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [sys.catalog]: This master's current role is: LEADER
I20251209 03:05:33.090111 9477 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [sys.catalog]: This master's current role is: LEADER
I20251209 03:05:33.090119 9483 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251209 03:05:33.090317 9483 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251209 03:05:33.090830 2731 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251209 03:05:33.090965 9483 catalog_manager.cc:1357] Generated new cluster ID: ea2fe4034b814d26b5443a77f7a10976
I20251209 03:05:33.091004 9483 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251209 03:05:33.112890 9483 catalog_manager.cc:1380] Generated new certificate authority record
I20251209 03:05:33.113377 9483 catalog_manager.cc:1514] Loading token signing keys...
I20251209 03:05:33.122428 9483 catalog_manager.cc:6027] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: Generated new TSK 0
I20251209 03:05:33.122535 9483 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251209 03:05:33.154747 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:33.156029 9494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:33.156108 9497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:33.156205 9495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:33.156318 2731 server_base.cc:1047] running on GCE node
I20251209 03:05:33.156423 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:33.156453 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:33.156489 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249533156490 us; error 0 us; skew 500 ppm
I20251209 03:05:33.157047 2731 webserver.cc:492] Webserver started at http://127.2.170.193:38939/ using document root <none> and password file <none>
I20251209 03:05:33.157122 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:33.157168 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:33.157214 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:33.157495 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root/instance:
uuid: "aa74703951b844639caf72b3bc20d475"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:33.158296 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:33.158710 9502 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.158824 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.158864 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "aa74703951b844639caf72b3bc20d475"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:33.158900 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:33.174018 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:33.174268 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:33.174440 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:33.174714 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:33.174752 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.174782 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:33.174806 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.178066 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:43401
I20251209 03:05:33.178146 9565 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:43401 every 8 connection(s)
I20251209 03:05:33.178480 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:33.179664 9573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:33.179772 9571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:33.179817 9570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:33.179956 2731 server_base.cc:1047] running on GCE node
I20251209 03:05:33.180202 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:33.180236 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:33.180248 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249533180248 us; error 0 us; skew 500 ppm
I20251209 03:05:33.180876 2731 webserver.cc:492] Webserver started at http://127.2.170.194:43147/ using document root <none> and password file <none>
I20251209 03:05:33.180948 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:33.180991 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:33.181036 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:33.181283 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root/instance:
uuid: "f106240450834d39a486b36a4ea76851"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:33.182047 9566 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41961
I20251209 03:05:33.182113 9566 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:33.182206 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:33.182228 9566 heartbeater.cc:507] Master 127.2.170.254:41961 requested a full tablet report, sending...
I20251209 03:05:33.182453 9437 ts_manager.cc:194] Registered new tserver with Master: aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401)
I20251209 03:05:33.182732 9578 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.182847 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.182893 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root
uuid: "f106240450834d39a486b36a4ea76851"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:33.182935 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:33.183087 9437 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59916
I20251209 03:05:33.201241 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:33.201444 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:33.201586 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:33.201815 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:33.201846 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.201869 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:33.201890 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.204610 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.194:38749
I20251209 03:05:33.204670 9641 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.194:38749 every 8 connection(s)
I20251209 03:05:33.204972 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251209 03:05:33.206307 9645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:33.206331 9646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:33.206454 9648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:33.206645 2731 server_base.cc:1047] running on GCE node
I20251209 03:05:33.206758 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:33.206804 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:33.206815 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249533206816 us; error 0 us; skew 500 ppm
I20251209 03:05:33.207581 2731 webserver.cc:492] Webserver started at http://127.2.170.195:33593/ using document root <none> and password file <none>
I20251209 03:05:33.207669 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:33.207715 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:33.207804 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:33.208128 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root/instance:
uuid: "5a68e1323b54428b9a4b841a924829cb"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:33.208216 9642 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41961
I20251209 03:05:33.208282 9642 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:33.208386 9642 heartbeater.cc:507] Master 127.2.170.254:41961 requested a full tablet report, sending...
I20251209 03:05:33.208621 9437 ts_manager.cc:194] Registered new tserver with Master: f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:33.209077 9437 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59924
I20251209 03:05:33.209122 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:33.209621 9653 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.209748 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.209795 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root
uuid: "5a68e1323b54428b9a4b841a924829cb"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:33.209839 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:33.215885 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:33.216064 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:33.216184 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:33.216346 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:33.216379 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.216405 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:33.216431 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:33.219033 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.195:44713
I20251209 03:05:33.219080 9716 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.195:44713 every 8 connection(s)
I20251209 03:05:33.221863 9717 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41961
I20251209 03:05:33.221923 9717 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:33.222013 9717 heartbeater.cc:507] Master 127.2.170.254:41961 requested a full tablet report, sending...
I20251209 03:05:33.222230 9437 ts_manager.cc:194] Registered new tserver with Master: 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:33.222527 2731 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.003328019s
I20251209 03:05:33.222582 9437 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59934
I20251209 03:05:34.183694 9566 heartbeater.cc:499] Master 127.2.170.254:41961 was elected leader, sending a full tablet report...
I20251209 03:05:34.209578 9642 heartbeater.cc:499] Master 127.2.170.254:41961 was elected leader, sending a full tablet report...
I20251209 03:05:34.223078 9717 heartbeater.cc:499] Master 127.2.170.254:41961 was elected leader, sending a full tablet report...
I20251209 03:05:34.247514 2731 test_util.cc:276] Using random seed: 795722327
I20251209 03:05:34.256776 9437 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59946:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:05:34.256968 9437 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:05:34.262676 9606 tablet_service.cc:1505] Processing CreateTablet for tablet 4135fcbfa27d4b438a22b2319f49bbd4 (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:34.262756 9603 tablet_service.cc:1505] Processing CreateTablet for tablet ed91b26a88e54442888350505b700d79 (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:34.262857 9606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4135fcbfa27d4b438a22b2319f49bbd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.262921 9604 tablet_service.cc:1505] Processing CreateTablet for tablet b2c804e735df449eab77a465a518f6ff (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:34.262961 9603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed91b26a88e54442888350505b700d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.263020 9604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c804e735df449eab77a465a518f6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.263119 9675 tablet_service.cc:1505] Processing CreateTablet for tablet ed91b26a88e54442888350505b700d79 (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:34.263206 9675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed91b26a88e54442888350505b700d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.263386 9680 tablet_service.cc:1505] Processing CreateTablet for tablet 2882613655374b0ab2fedbfb0676349d (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:34.263466 9680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2882613655374b0ab2fedbfb0676349d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.264395 9679 tablet_service.cc:1505] Processing CreateTablet for tablet b2c804e735df449eab77a465a518f6ff (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:34.264482 9737 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: Bootstrap starting.
I20251209 03:05:34.264475 9605 tablet_service.cc:1505] Processing CreateTablet for tablet 2882613655374b0ab2fedbfb0676349d (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:34.264607 9605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2882613655374b0ab2fedbfb0676349d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.264798 9681 tablet_service.cc:1505] Processing CreateTablet for tablet 4135fcbfa27d4b438a22b2319f49bbd4 (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:34.264935 9737 tablet_bootstrap.cc:654] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.264495 9679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c804e735df449eab77a465a518f6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.265694 9681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4135fcbfa27d4b438a22b2319f49bbd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.265940 9530 tablet_service.cc:1505] Processing CreateTablet for tablet 4135fcbfa27d4b438a22b2319f49bbd4 (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:34.266000 9527 tablet_service.cc:1505] Processing CreateTablet for tablet ed91b26a88e54442888350505b700d79 (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:34.266055 9530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4135fcbfa27d4b438a22b2319f49bbd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.266103 9528 tablet_service.cc:1505] Processing CreateTablet for tablet b2c804e735df449eab77a465a518f6ff (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:34.266181 9528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c804e735df449eab77a465a518f6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.267133 9737 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: No bootstrap required, opened a new log
I20251209 03:05:34.267232 9737 ts_tablet_manager.cc:1403] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251209 03:05:34.267377 9529 tablet_service.cc:1505] Processing CreateTablet for tablet 2882613655374b0ab2fedbfb0676349d (DEFAULT_TABLE table=test-workload [id=c433a9ff555e4c788994a06208f4ccb0]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:34.267432 9740 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:34.267514 9529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2882613655374b0ab2fedbfb0676349d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.267553 9737 raft_consensus.cc:359] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.267691 9737 raft_consensus.cc:385] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.267751 9737 raft_consensus.cc:740] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f106240450834d39a486b36a4ea76851, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.267863 9737 consensus_queue.cc:260] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.267971 9740 tablet_bootstrap.cc:654] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.268206 9738 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: Bootstrap starting.
I20251209 03:05:34.268462 9737 ts_tablet_manager.cc:1434] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:34.268569 9737 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851: Bootstrap starting.
I20251209 03:05:34.268654 9738 tablet_bootstrap.cc:654] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.268689 9527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed91b26a88e54442888350505b700d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.269006 9737 tablet_bootstrap.cc:654] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.269335 9740 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: No bootstrap required, opened a new log
I20251209 03:05:34.269395 9738 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: No bootstrap required, opened a new log
I20251209 03:05:34.269388 9740 ts_tablet_manager.cc:1403] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:34.269472 9738 ts_tablet_manager.cc:1403] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.269634 9737 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851: No bootstrap required, opened a new log
I20251209 03:05:34.269655 9740 raft_consensus.cc:359] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.269690 9738 raft_consensus.cc:359] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.269771 9738 raft_consensus.cc:385] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.269762 9740 raft_consensus.cc:385] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.269807 9738 raft_consensus.cc:740] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.269822 9740 raft_consensus.cc:740] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.269851 9738 consensus_queue.cc:260] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.269860 9740 consensus_queue.cc:260] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.269989 9738 ts_tablet_manager.cc:1434] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.270036 9740 ts_tablet_manager.cc:1434] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:34.270044 9738 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb: Bootstrap starting.
I20251209 03:05:34.270100 9740 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:34.269757 9737 ts_tablet_manager.cc:1403] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.270466 9738 tablet_bootstrap.cc:654] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.270503 9740 tablet_bootstrap.cc:654] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.270493 9737 raft_consensus.cc:359] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.270558 9737 raft_consensus.cc:385] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.270577 9737 raft_consensus.cc:740] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f106240450834d39a486b36a4ea76851, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.270618 9737 consensus_queue.cc:260] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.270689 9737 ts_tablet_manager.cc:1434] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.270735 9737 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851: Bootstrap starting.
I20251209 03:05:34.271075 9737 tablet_bootstrap.cc:654] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.271212 9738 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb: No bootstrap required, opened a new log
I20251209 03:05:34.271258 9738 ts_tablet_manager.cc:1403] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.271389 9738 raft_consensus.cc:359] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.271418 9740 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: No bootstrap required, opened a new log
I20251209 03:05:34.271458 9738 raft_consensus.cc:385] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.271481 9738 raft_consensus.cc:740] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.271481 9740 ts_tablet_manager.cc:1403] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:34.271525 9738 consensus_queue.cc:260] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.271600 9738 ts_tablet_manager.cc:1434] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.271657 9738 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb: Bootstrap starting.
I20251209 03:05:34.271654 9740 raft_consensus.cc:359] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.271709 9737 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851: No bootstrap required, opened a new log
I20251209 03:05:34.271728 9740 raft_consensus.cc:385] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.271749 9740 raft_consensus.cc:740] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.271749 9737 ts_tablet_manager.cc:1403] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.271798 9740 consensus_queue.cc:260] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.271875 9740 ts_tablet_manager.cc:1434] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.271899 9737 raft_consensus.cc:359] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.271926 9740 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:34.271941 9737 raft_consensus.cc:385] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.271960 9737 raft_consensus.cc:740] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f106240450834d39a486b36a4ea76851, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.271994 9737 consensus_queue.cc:260] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.272066 9737 ts_tablet_manager.cc:1434] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.272105 9738 tablet_bootstrap.cc:654] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.272114 9737 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851: Bootstrap starting.
I20251209 03:05:34.272297 9740 tablet_bootstrap.cc:654] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.272601 9738 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb: No bootstrap required, opened a new log
I20251209 03:05:34.272657 9738 ts_tablet_manager.cc:1403] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.272607 9737 tablet_bootstrap.cc:654] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.272778 9738 raft_consensus.cc:359] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.272837 9738 raft_consensus.cc:385] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.272861 9738 raft_consensus.cc:740] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.272897 9738 consensus_queue.cc:260] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.273082 9738 ts_tablet_manager.cc:1434] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.273152 9738 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb: Bootstrap starting.
I20251209 03:05:34.273499 9738 tablet_bootstrap.cc:654] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.273991 9740 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: No bootstrap required, opened a new log
I20251209 03:05:34.274048 9737 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851: No bootstrap required, opened a new log
I20251209 03:05:34.274039 9740 ts_tablet_manager.cc:1403] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251209 03:05:34.274044 9738 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb: No bootstrap required, opened a new log
I20251209 03:05:34.274106 9737 ts_tablet_manager.cc:1403] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:34.274120 9738 ts_tablet_manager.cc:1403] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:34.274191 9740 raft_consensus.cc:359] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.274238 9740 raft_consensus.cc:385] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.274250 9740 raft_consensus.cc:740] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.274287 9740 consensus_queue.cc:260] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.274256 9738 raft_consensus.cc:359] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.274255 9737 raft_consensus.cc:359] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.274353 9738 raft_consensus.cc:385] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.274364 9737 raft_consensus.cc:385] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.274384 9737 raft_consensus.cc:740] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f106240450834d39a486b36a4ea76851, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.274372 9738 raft_consensus.cc:740] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.274380 9740 ts_tablet_manager.cc:1434] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.274420 9737 consensus_queue.cc:260] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.274500 9740 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:34.274441 9738 consensus_queue.cc:260] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.274549 9737 ts_tablet_manager.cc:1434] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.274605 9738 ts_tablet_manager.cc:1434] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.274895 9740 tablet_bootstrap.cc:654] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.275518 9740 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: No bootstrap required, opened a new log
I20251209 03:05:34.275566 9740 ts_tablet_manager.cc:1403] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:34.275698 9740 raft_consensus.cc:359] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.275743 9740 raft_consensus.cc:385] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.275759 9740 raft_consensus.cc:740] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.275806 9740 consensus_queue.cc:260] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.275920 9740 ts_tablet_manager.cc:1434] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.277323 9745 raft_consensus.cc:493] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:34.277371 9745 raft_consensus.cc:515] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.277518 9745 leader_election.cc:290] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f106240450834d39a486b36a4ea76851 (127.2.170.194:38749), aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401)
I20251209 03:05:34.280014 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4135fcbfa27d4b438a22b2319f49bbd4" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475" is_pre_election: true
I20251209 03:05:34.280081 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4135fcbfa27d4b438a22b2319f49bbd4" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851" is_pre_election: true
I20251209 03:05:34.280113 9539 raft_consensus.cc:2468] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 0.
I20251209 03:05:34.280150 9616 raft_consensus.cc:2468] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 0.
I20251209 03:05:34.280277 9657 leader_election.cc:304] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475; no voters:
I20251209 03:05:34.280364 9745 raft_consensus.cc:2804] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:34.280405 9745 raft_consensus.cc:493] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:34.280426 9745 raft_consensus.cc:3060] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.280939 9745 raft_consensus.cc:515] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.281033 9745 leader_election.cc:290] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 election: Requested vote from peers f106240450834d39a486b36a4ea76851 (127.2.170.194:38749), aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401)
I20251209 03:05:34.281185 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4135fcbfa27d4b438a22b2319f49bbd4" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475"
I20251209 03:05:34.281235 9539 raft_consensus.cc:3060] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.281213 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4135fcbfa27d4b438a22b2319f49bbd4" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851"
I20251209 03:05:34.281270 9616 raft_consensus.cc:3060] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.281792 9539 raft_consensus.cc:2468] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:34.281945 9657 leader_election.cc:304] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475; no voters:
I20251209 03:05:34.281991 9616 raft_consensus.cc:2468] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:34.282011 9745 raft_consensus.cc:2804] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:34.282140 9745 raft_consensus.cc:697] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Becoming Leader. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Running, Role: LEADER
I20251209 03:05:34.282188 9745 consensus_queue.cc:237] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.282655 9437 catalog_manager.cc:5654] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a68e1323b54428b9a4b841a924829cb (127.2.170.195). New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } }
I20251209 03:05:34.350464 9741 raft_consensus.cc:493] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:34.350569 9741 raft_consensus.cc:515] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.350777 9741 leader_election.cc:290] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:34.353372 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2882613655374b0ab2fedbfb0676349d" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475" is_pre_election: true
I20251209 03:05:34.353483 9539 raft_consensus.cc:2468] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 0.
I20251209 03:05:34.353449 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2882613655374b0ab2fedbfb0676349d" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb" is_pre_election: true
I20251209 03:05:34.353535 9691 raft_consensus.cc:2468] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 0.
I20251209 03:05:34.353626 9582 leader_election.cc:304] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa74703951b844639caf72b3bc20d475, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:34.353746 9741 raft_consensus.cc:2804] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:34.353781 9741 raft_consensus.cc:493] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:34.353801 9741 raft_consensus.cc:3060] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.354409 9741 raft_consensus.cc:515] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.354537 9741 leader_election.cc:290] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 election: Requested vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:34.354692 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2882613655374b0ab2fedbfb0676349d" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb"
I20251209 03:05:34.354692 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2882613655374b0ab2fedbfb0676349d" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475"
I20251209 03:05:34.354779 9691 raft_consensus.cc:3060] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.354784 9539 raft_consensus.cc:3060] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.355397 9539 raft_consensus.cc:2468] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 1.
I20251209 03:05:34.355397 9691 raft_consensus.cc:2468] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 1.
I20251209 03:05:34.355557 9580 leader_election.cc:304] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:34.355623 9741 raft_consensus.cc:2804] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:34.355773 9741 raft_consensus.cc:697] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Becoming Leader. State: Replica: f106240450834d39a486b36a4ea76851, State: Running, Role: LEADER
I20251209 03:05:34.355826 9741 consensus_queue.cc:237] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.356362 9437 catalog_manager.cc:5654] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 reported cstate change: term changed from 0 to 1, leader changed from <none> to f106240450834d39a486b36a4ea76851 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "f106240450834d39a486b36a4ea76851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:34.366827 9746 raft_consensus.cc:493] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:34.366905 9746 raft_consensus.cc:515] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.367089 9746 leader_election.cc:290] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f106240450834d39a486b36a4ea76851 (127.2.170.194:38749), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:34.369542 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "aa74703951b844639caf72b3bc20d475" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851" is_pre_election: true
I20251209 03:05:34.369562 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "aa74703951b844639caf72b3bc20d475" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb" is_pre_election: true
I20251209 03:05:34.369635 9616 raft_consensus.cc:2468] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa74703951b844639caf72b3bc20d475 in term 0.
I20251209 03:05:34.369642 9691 raft_consensus.cc:2468] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa74703951b844639caf72b3bc20d475 in term 0.
I20251209 03:05:34.369822 9504 leader_election.cc:304] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475; no voters:
I20251209 03:05:34.369915 9746 raft_consensus.cc:2804] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:34.369948 9746 raft_consensus.cc:493] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:34.369961 9746 raft_consensus.cc:3060] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.370731 9746 raft_consensus.cc:515] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.370867 9746 leader_election.cc:290] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [CANDIDATE]: Term 1 election: Requested vote from peers f106240450834d39a486b36a4ea76851 (127.2.170.194:38749), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:34.370997 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "aa74703951b844639caf72b3bc20d475" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb"
I20251209 03:05:34.371042 9691 raft_consensus.cc:3060] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.371548 9691 raft_consensus.cc:2468] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa74703951b844639caf72b3bc20d475 in term 1.
I20251209 03:05:34.371636 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "aa74703951b844639caf72b3bc20d475" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851"
I20251209 03:05:34.371688 9616 raft_consensus.cc:3060] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.371690 9504 leader_election.cc:304] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475; no voters:
I20251209 03:05:34.371799 9746 raft_consensus.cc:2804] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:34.371934 9746 raft_consensus.cc:697] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 LEADER]: Becoming Leader. State: Replica: aa74703951b844639caf72b3bc20d475, State: Running, Role: LEADER
I20251209 03:05:34.371994 9746 consensus_queue.cc:237] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.372243 9616 raft_consensus.cc:2468] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa74703951b844639caf72b3bc20d475 in term 1.
I20251209 03:05:34.372488 9437 catalog_manager.cc:5654] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa74703951b844639caf72b3bc20d475 (127.2.170.193). New cstate: current_term: 1 leader_uuid: "aa74703951b844639caf72b3bc20d475" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:34.725389 9745 raft_consensus.cc:493] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:34.725497 9745 raft_consensus.cc:515] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.725656 9745 leader_election.cc:290] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:34.725916 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b2c804e735df449eab77a465a518f6ff" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851" is_pre_election: true
I20251209 03:05:34.725994 9616 raft_consensus.cc:2468] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 0.
I20251209 03:05:34.725917 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b2c804e735df449eab77a465a518f6ff" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475" is_pre_election: true
I20251209 03:05:34.726065 9539 raft_consensus.cc:2468] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 0.
I20251209 03:05:34.726169 9654 leader_election.cc:304] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:34.726253 9745 raft_consensus.cc:2804] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:34.726294 9745 raft_consensus.cc:493] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:34.726316 9745 raft_consensus.cc:3060] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.726946 9745 raft_consensus.cc:515] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.727074 9745 leader_election.cc:290] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 election: Requested vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:34.727182 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b2c804e735df449eab77a465a518f6ff" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475"
I20251209 03:05:34.727197 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b2c804e735df449eab77a465a518f6ff" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851"
I20251209 03:05:34.727236 9539 raft_consensus.cc:3060] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.727239 9616 raft_consensus.cc:3060] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.727866 9539 raft_consensus.cc:2468] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:34.727866 9616 raft_consensus.cc:2468] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:34.728040 9654 leader_election.cc:304] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:34.728127 9745 raft_consensus.cc:2804] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:34.728178 9745 raft_consensus.cc:697] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Becoming Leader. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Running, Role: LEADER
I20251209 03:05:34.728214 9745 consensus_queue.cc:237] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.728757 9437 catalog_manager.cc:5654] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a68e1323b54428b9a4b841a924829cb (127.2.170.195). New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } }
I20251209 03:05:34.749547 9746 consensus_queue.cc:1048] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:34.751199 9771 consensus_queue.cc:1048] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:34.793648 9741 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:34.795068 9741 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:34.811407 2731 test_util.cc:276] Using random seed: 796286213
I20251209 03:05:34.811749 9745 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:34.813282 9757 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:34.816461 9436 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59962:
name: "dugtrio"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251209 03:05:34.816606 9436 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table dugtrio in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251209 03:05:34.817997 9529 tablet_service.cc:1505] Processing CreateTablet for tablet acfa3c3152b84ba1973e14357cd67fc6 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:34.818117 9605 tablet_service.cc:1505] Processing CreateTablet for tablet acfa3c3152b84ba1973e14357cd67fc6 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:34.818195 9527 tablet_service.cc:1505] Processing CreateTablet for tablet 09cd21d9bc7a4eb199bf120c0892a077 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:34.818193 9603 tablet_service.cc:1505] Processing CreateTablet for tablet 09cd21d9bc7a4eb199bf120c0892a077 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:34.818193 9606 tablet_service.cc:1505] Processing CreateTablet for tablet 9de8c0db7da244ba8bdeecbc3350bba2 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:34.818363 9679 tablet_service.cc:1505] Processing CreateTablet for tablet 09cd21d9bc7a4eb199bf120c0892a077 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20251209 03:05:34.818377 9530 tablet_service.cc:1505] Processing CreateTablet for tablet cac190172b0144b3a3c3b9e1de80dcab (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:34.818403 9604 tablet_service.cc:1505] Processing CreateTablet for tablet cac190172b0144b3a3c3b9e1de80dcab (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:34.818431 9679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cd21d9bc7a4eb199bf120c0892a077. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.818240 9605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acfa3c3152b84ba1973e14357cd67fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.818492 9528 tablet_service.cc:1505] Processing CreateTablet for tablet 9de8c0db7da244ba8bdeecbc3350bba2 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:34.818624 9681 tablet_service.cc:1505] Processing CreateTablet for tablet 9de8c0db7da244ba8bdeecbc3350bba2 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20251209 03:05:34.818707 9681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de8c0db7da244ba8bdeecbc3350bba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.819842 9675 tablet_service.cc:1505] Processing CreateTablet for tablet acfa3c3152b84ba1973e14357cd67fc6 (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20251209 03:05:34.819929 9675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acfa3c3152b84ba1973e14357cd67fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.818512 9603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cd21d9bc7a4eb199bf120c0892a077. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.820204 9604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac190172b0144b3a3c3b9e1de80dcab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.818125 9529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acfa3c3152b84ba1973e14357cd67fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.821301 9527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cd21d9bc7a4eb199bf120c0892a077. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.821415 9606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de8c0db7da244ba8bdeecbc3350bba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.821723 9528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de8c0db7da244ba8bdeecbc3350bba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.822474 9784 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: Bootstrap starting.
I20251209 03:05:34.822489 9786 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:34.822805 9681 tablet_service.cc:1505] Processing CreateTablet for tablet cac190172b0144b3a3c3b9e1de80dcab (DEFAULT_TABLE table=dugtrio [id=4869122f1d3645f09c04569da8a692a2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20251209 03:05:34.822880 9784 tablet_bootstrap.cc:654] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.822911 9681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac190172b0144b3a3c3b9e1de80dcab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.823122 9786 tablet_bootstrap.cc:654] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.823640 9786 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: No bootstrap required, opened a new log
I20251209 03:05:34.823693 9786 ts_tablet_manager.cc:1403] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.823845 9786 raft_consensus.cc:359] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.823918 9786 raft_consensus.cc:385] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.823947 9786 raft_consensus.cc:740] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.822803 9530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac190172b0144b3a3c3b9e1de80dcab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:34.823988 9786 consensus_queue.cc:260] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.824182 9786 ts_tablet_manager.cc:1434] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:34.824255 9786 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:34.824266 9784 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: No bootstrap required, opened a new log
I20251209 03:05:34.824334 9784 ts_tablet_manager.cc:1403] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:34.824513 9784 raft_consensus.cc:359] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.824572 9784 raft_consensus.cc:385] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.824592 9784 raft_consensus.cc:740] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.824657 9784 consensus_queue.cc:260] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.824677 9786 tablet_bootstrap.cc:654] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.824855 9784 ts_tablet_manager.cc:1434] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.824916 9784 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: Bootstrap starting.
I20251209 03:05:34.825268 9786 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: No bootstrap required, opened a new log
I20251209 03:05:34.825302 9784 tablet_bootstrap.cc:654] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.825309 9786 ts_tablet_manager.cc:1403] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.825469 9786 raft_consensus.cc:359] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.825520 9786 raft_consensus.cc:385] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.825539 9786 raft_consensus.cc:740] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.825594 9786 consensus_queue.cc:260] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.825729 9786 ts_tablet_manager.cc:1434] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.825790 9786 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:34.825995 9785 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: Bootstrap starting.
I20251209 03:05:34.826174 9786 tablet_bootstrap.cc:654] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.826467 9784 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: No bootstrap required, opened a new log
I20251209 03:05:34.826516 9784 ts_tablet_manager.cc:1403] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:34.826661 9784 raft_consensus.cc:359] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.826712 9784 raft_consensus.cc:385] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.826730 9784 raft_consensus.cc:740] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.826802 9785 tablet_bootstrap.cc:654] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.826797 9784 consensus_queue.cc:260] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.827003 9784 ts_tablet_manager.cc:1434] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:34.827088 9784 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: Bootstrap starting.
I20251209 03:05:34.827019 9786 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: No bootstrap required, opened a new log
I20251209 03:05:34.827217 9786 ts_tablet_manager.cc:1403] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.827353 9786 raft_consensus.cc:359] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.827476 9784 tablet_bootstrap.cc:654] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.827410 9786 raft_consensus.cc:385] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.827526 9786 raft_consensus.cc:740] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.827600 9785 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: No bootstrap required, opened a new log
I20251209 03:05:34.827606 9786 consensus_queue.cc:260] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.827654 9785 ts_tablet_manager.cc:1403] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:34.827823 9785 raft_consensus.cc:359] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.827889 9785 raft_consensus.cc:385] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.827913 9785 raft_consensus.cc:740] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f106240450834d39a486b36a4ea76851, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.827950 9785 consensus_queue.cc:260] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.828022 9784 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: No bootstrap required, opened a new log
I20251209 03:05:34.828035 9785 ts_tablet_manager.cc:1434] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.828069 9784 ts_tablet_manager.cc:1403] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.828127 9785 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: Bootstrap starting.
I20251209 03:05:34.828204 9784 raft_consensus.cc:359] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.828250 9784 raft_consensus.cc:385] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.828261 9784 raft_consensus.cc:740] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.828294 9784 consensus_queue.cc:260] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.828365 9784 ts_tablet_manager.cc:1434] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.828415 9784 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: Bootstrap starting.
I20251209 03:05:34.828545 9785 tablet_bootstrap.cc:654] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.828779 9784 tablet_bootstrap.cc:654] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.828939 9786 ts_tablet_manager.cc:1434] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251209 03:05:34.829005 9786 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:34.829391 9786 tablet_bootstrap.cc:654] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.829480 9785 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: No bootstrap required, opened a new log
I20251209 03:05:34.829528 9785 ts_tablet_manager.cc:1403] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.829727 9785 raft_consensus.cc:359] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.829790 9785 raft_consensus.cc:385] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.829805 9784 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: No bootstrap required, opened a new log
I20251209 03:05:34.829864 9784 ts_tablet_manager.cc:1403] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.829808 9785 raft_consensus.cc:740] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f106240450834d39a486b36a4ea76851, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.829936 9785 consensus_queue.cc:260] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.829972 9786 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: No bootstrap required, opened a new log
I20251209 03:05:34.830004 9786 ts_tablet_manager.cc:1403] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:34.830040 9785 ts_tablet_manager.cc:1434] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.830018 9784 raft_consensus.cc:359] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.830094 9784 raft_consensus.cc:385] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.830116 9784 raft_consensus.cc:740] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.830116 9785 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: Bootstrap starting.
I20251209 03:05:34.830168 9786 raft_consensus.cc:359] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.830211 9786 raft_consensus.cc:385] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.830199 9784 consensus_queue.cc:260] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.830229 9786 raft_consensus.cc:740] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.830294 9786 consensus_queue.cc:260] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.830296 9784 ts_tablet_manager.cc:1434] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.830540 9785 tablet_bootstrap.cc:654] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.830684 9786 ts_tablet_manager.cc:1434] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:34.831282 9785 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: No bootstrap required, opened a new log
I20251209 03:05:34.831322 9785 ts_tablet_manager.cc:1403] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.831434 9785 raft_consensus.cc:359] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.831475 9785 raft_consensus.cc:385] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.831485 9785 raft_consensus.cc:740] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f106240450834d39a486b36a4ea76851, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.831521 9785 consensus_queue.cc:260] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.831596 9785 ts_tablet_manager.cc:1434] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.831647 9785 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: Bootstrap starting.
I20251209 03:05:34.832000 9785 tablet_bootstrap.cc:654] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: Neither blocks nor log segments found. Creating new log.
I20251209 03:05:34.832526 9785 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: No bootstrap required, opened a new log
I20251209 03:05:34.832566 9785 ts_tablet_manager.cc:1403] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:34.832674 9785 raft_consensus.cc:359] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.832712 9785 raft_consensus.cc:385] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251209 03:05:34.832724 9785 raft_consensus.cc:740] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f106240450834d39a486b36a4ea76851, State: Initialized, Role: FOLLOWER
I20251209 03:05:34.832759 9785 consensus_queue.cc:260] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.832837 9785 ts_tablet_manager.cc:1434] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:34.893055 9757 raft_consensus.cc:493] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:34.893163 9757 raft_consensus.cc:515] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.893328 9757 leader_election.cc:290] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:34.893515 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9de8c0db7da244ba8bdeecbc3350bba2" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851" is_pre_election: true
I20251209 03:05:34.893522 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9de8c0db7da244ba8bdeecbc3350bba2" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475" is_pre_election: true
I20251209 03:05:34.893582 9616 raft_consensus.cc:2468] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 0.
I20251209 03:05:34.893631 9539 raft_consensus.cc:2468] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 0.
I20251209 03:05:34.893764 9654 leader_election.cc:304] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:34.893853 9757 raft_consensus.cc:2804] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:34.893899 9757 raft_consensus.cc:493] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:34.893913 9757 raft_consensus.cc:3060] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.894619 9757 raft_consensus.cc:515] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.894735 9757 leader_election.cc:290] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 election: Requested vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:34.894861 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9de8c0db7da244ba8bdeecbc3350bba2" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475"
I20251209 03:05:34.894861 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9de8c0db7da244ba8bdeecbc3350bba2" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851"
I20251209 03:05:34.894940 9539 raft_consensus.cc:3060] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.894949 9616 raft_consensus.cc:3060] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.895576 9539 raft_consensus.cc:2468] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:34.895576 9616 raft_consensus.cc:2468] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:34.895749 9654 leader_election.cc:304] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:34.895854 9757 raft_consensus.cc:2804] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:34.895903 9757 raft_consensus.cc:697] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Becoming Leader. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Running, Role: LEADER
I20251209 03:05:34.895951 9757 consensus_queue.cc:237] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.896426 9435 catalog_manager.cc:5654] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a68e1323b54428b9a4b841a924829cb (127.2.170.195). New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } }
I20251209 03:05:34.983566 9741 raft_consensus.cc:493] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:34.983671 9741 raft_consensus.cc:515] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.983810 9741 leader_election.cc:290] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:34.984033 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cac190172b0144b3a3c3b9e1de80dcab" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb" is_pre_election: true
I20251209 03:05:34.984107 9691 raft_consensus.cc:2468] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 0.
I20251209 03:05:34.984037 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cac190172b0144b3a3c3b9e1de80dcab" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475" is_pre_election: true
I20251209 03:05:34.984154 9539 raft_consensus.cc:2468] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 0.
I20251209 03:05:34.984261 9580 leader_election.cc:304] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:34.984349 9741 raft_consensus.cc:2804] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:34.984390 9741 raft_consensus.cc:493] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:34.984409 9741 raft_consensus.cc:3060] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.985028 9741 raft_consensus.cc:515] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.985139 9741 leader_election.cc:290] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 election: Requested vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:34.985307 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cac190172b0144b3a3c3b9e1de80dcab" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb"
I20251209 03:05:34.985308 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cac190172b0144b3a3c3b9e1de80dcab" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475"
I20251209 03:05:34.985428 9539 raft_consensus.cc:3060] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.985419 9691 raft_consensus.cc:3060] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.986151 9691 raft_consensus.cc:2468] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 1.
I20251209 03:05:34.986148 9539 raft_consensus.cc:2468] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 1.
I20251209 03:05:34.986342 9582 leader_election.cc:304] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa74703951b844639caf72b3bc20d475, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:34.986434 9741 raft_consensus.cc:2804] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:34.986485 9741 raft_consensus.cc:697] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Becoming Leader. State: Replica: f106240450834d39a486b36a4ea76851, State: Running, Role: LEADER
I20251209 03:05:34.986527 9741 consensus_queue.cc:237] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.987044 9435 catalog_manager.cc:5654] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 reported cstate change: term changed from 0 to 1, leader changed from <none> to f106240450834d39a486b36a4ea76851 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "f106240450834d39a486b36a4ea76851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:34.991075 9757 raft_consensus.cc:493] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:34.991132 9757 raft_consensus.cc:515] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.991254 9757 leader_election.cc:290] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:34.991410 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acfa3c3152b84ba1973e14357cd67fc6" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475" is_pre_election: true
I20251209 03:05:34.991478 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acfa3c3152b84ba1973e14357cd67fc6" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851" is_pre_election: true
I20251209 03:05:34.991487 9539 raft_consensus.cc:2468] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 0.
I20251209 03:05:34.991544 9616 raft_consensus.cc:2468] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 0.
I20251209 03:05:34.991636 9657 leader_election.cc:304] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475; no voters:
I20251209 03:05:34.991711 9757 raft_consensus.cc:2804] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:34.991755 9757 raft_consensus.cc:493] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:34.991775 9757 raft_consensus.cc:3060] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.992333 9757 raft_consensus.cc:515] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.992448 9757 leader_election.cc:290] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 election: Requested vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:34.992558 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acfa3c3152b84ba1973e14357cd67fc6" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475"
I20251209 03:05:34.992576 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "acfa3c3152b84ba1973e14357cd67fc6" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851"
I20251209 03:05:34.992656 9539 raft_consensus.cc:3060] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.992676 9616 raft_consensus.cc:3060] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:34.993256 9616 raft_consensus.cc:2468] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:34.993256 9539 raft_consensus.cc:2468] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:34.993429 9657 leader_election.cc:304] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475; no voters:
I20251209 03:05:34.993491 9757 raft_consensus.cc:2804] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:34.993547 9757 raft_consensus.cc:697] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Becoming Leader. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Running, Role: LEADER
I20251209 03:05:34.993584 9757 consensus_queue.cc:237] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:34.994107 9435 catalog_manager.cc:5654] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a68e1323b54428b9a4b841a924829cb (127.2.170.195). New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } }
W20251209 03:05:35.090587 9486 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20251209 03:05:35.188261 9741 raft_consensus.cc:493] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251209 03:05:35.188367 9741 raft_consensus.cc:515] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:35.188529 9741 leader_election.cc:290] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:35.188723 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09cd21d9bc7a4eb199bf120c0892a077" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475" is_pre_election: true
I20251209 03:05:35.188720 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09cd21d9bc7a4eb199bf120c0892a077" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb" is_pre_election: true
I20251209 03:05:35.188818 9539 raft_consensus.cc:2468] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 0.
I20251209 03:05:35.188831 9691 raft_consensus.cc:2468] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 0.
I20251209 03:05:35.188980 9580 leader_election.cc:304] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:35.189079 9741 raft_consensus.cc:2804] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251209 03:05:35.189112 9741 raft_consensus.cc:493] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251209 03:05:35.189131 9741 raft_consensus.cc:3060] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:35.189783 9741 raft_consensus.cc:515] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:35.189908 9741 leader_election.cc:290] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 election: Requested vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:35.190040 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09cd21d9bc7a4eb199bf120c0892a077" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb"
I20251209 03:05:35.190039 9539 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09cd21d9bc7a4eb199bf120c0892a077" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475"
I20251209 03:05:35.190112 9691 raft_consensus.cc:3060] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:35.190125 9539 raft_consensus.cc:3060] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 0 FOLLOWER]: Advancing to term 1
I20251209 03:05:35.190653 9539 raft_consensus.cc:2468] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 1.
I20251209 03:05:35.190757 9691 raft_consensus.cc:2468] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f106240450834d39a486b36a4ea76851 in term 1.
I20251209 03:05:35.190825 9582 leader_election.cc:304] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa74703951b844639caf72b3bc20d475, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:35.190901 9741 raft_consensus.cc:2804] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader election won for term 1
I20251209 03:05:35.190956 9741 raft_consensus.cc:697] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Becoming Leader. State: Replica: f106240450834d39a486b36a4ea76851, State: Running, Role: LEADER
I20251209 03:05:35.191006 9741 consensus_queue.cc:237] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:35.191551 9435 catalog_manager.cc:5654] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 reported cstate change: term changed from 0 to 1, leader changed from <none> to f106240450834d39a486b36a4ea76851 (127.2.170.194). New cstate: current_term: 1 leader_uuid: "f106240450834d39a486b36a4ea76851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:35.250046 9745 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:35.251663 9804 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:35.261314 2731 tablet_server.cc:178] TabletServer@127.2.170.193:0 shutting down...
I20251209 03:05:35.264171 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:35.264382 2731 tablet_replica.cc:333] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: stopping tablet replica
I20251209 03:05:35.264457 2731 raft_consensus.cc:2243] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:35.264505 2731 raft_consensus.cc:2272] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:35.264707 2731 tablet_replica.cc:333] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: stopping tablet replica
I20251209 03:05:35.264755 2731 raft_consensus.cc:2243] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:35.264793 2731 raft_consensus.cc:2272] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:35.264984 2731 tablet_replica.cc:333] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: stopping tablet replica
I20251209 03:05:35.265025 2731 raft_consensus.cc:2243] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:35.265059 2731 raft_consensus.cc:2272] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:35.265215 2731 tablet_replica.cc:333] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: stopping tablet replica
I20251209 03:05:35.265259 2731 raft_consensus.cc:2243] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:35.265286 2731 raft_consensus.cc:2272] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:35.265462 2731 tablet_replica.cc:333] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: stopping tablet replica
I20251209 03:05:35.265504 2731 raft_consensus.cc:2243] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:35.265537 2731 raft_consensus.cc:2272] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:35.265738 2731 tablet_replica.cc:333] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: stopping tablet replica
I20251209 03:05:35.265803 2731 raft_consensus.cc:2243] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:35.265864 2731 raft_consensus.cc:2272] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:35.266048 2731 tablet_replica.cc:333] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: stopping tablet replica
I20251209 03:05:35.266083 2731 raft_consensus.cc:2243] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:35.266110 2731 raft_consensus.cc:2272] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:35.266266 2731 tablet_replica.cc:333] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: stopping tablet replica
I20251209 03:05:35.266299 2731 raft_consensus.cc:2243] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:35.266324 2731 raft_consensus.cc:2272] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:35.278317 2731 tablet_server.cc:195] TabletServer@127.2.170.193:0 shutdown complete.
W20251209 03:05:35.313467 9582 consensus_peers.cc:597] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 -> Peer aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): Couldn't send request to peer aa74703951b844639caf72b3bc20d475. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:35.367795 9804 consensus_queue.cc:1048] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251209 03:05:35.369508 9657 consensus_peers.cc:597] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb -> Peer aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): Couldn't send request to peer aa74703951b844639caf72b3bc20d475. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251209 03:05:35.376029 9657 consensus_peers.cc:597] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb -> Peer aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): Couldn't send request to peer aa74703951b844639caf72b3bc20d475. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251209 03:05:35.398039 9657 consensus_peers.cc:597] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb -> Peer aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): Couldn't send request to peer aa74703951b844639caf72b3bc20d475. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251209 03:05:35.403311 9582 consensus_peers.cc:597] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 -> Peer aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): Couldn't send request to peer aa74703951b844639caf72b3bc20d475. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:35.488705 9761 consensus_queue.cc:1048] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:35.598443 9745 consensus_queue.cc:1048] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251209 03:05:35.729179 9741 consensus_queue.cc:1048] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251209 03:05:35.730785 9582 consensus_peers.cc:597] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 -> Peer aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): Couldn't send request to peer aa74703951b844639caf72b3bc20d475. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:35.802248 9814 consensus_queue.cc:579] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Leader has been unable to successfully communicate with peer aa74703951b844639caf72b3bc20d475 for more than 1 seconds (1.006s)
W20251209 03:05:35.836541 9657 consensus_peers.cc:597] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb -> Peer aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): Couldn't send request to peer aa74703951b844639caf72b3bc20d475. Status: Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:36.163101 9757 consensus_queue.cc:579] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Leader has been unable to successfully communicate with peer aa74703951b844639caf72b3bc20d475 for more than 1 seconds (1.350s)
I20251209 03:05:36.185039 9804 consensus_queue.cc:579] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Leader has been unable to successfully communicate with peer aa74703951b844639caf72b3bc20d475 for more than 1 seconds (1.289s)
I20251209 03:05:36.252287 9814 raft_consensus.cc:493] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa74703951b844639caf72b3bc20d475)
I20251209 03:05:36.252400 9814 raft_consensus.cc:515] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.252574 9814 leader_election.cc:290] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:36.252784 9691 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "f106240450834d39a486b36a4ea76851" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a68e1323b54428b9a4b841a924829cb" is_pre_election: true
W20251209 03:05:36.253086 9582 leader_election.cc:336] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111)
I20251209 03:05:36.253150 9582 leader_election.cc:304] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f106240450834d39a486b36a4ea76851; no voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475
I20251209 03:05:36.253229 9814 raft_consensus.cc:2749] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251209 03:05:36.281244 9757 consensus_queue.cc:579] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Leader has been unable to successfully communicate with peer aa74703951b844639caf72b3bc20d475 for more than 1 seconds (1.288s)
I20251209 03:05:36.282856 9435 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59946:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251209 03:05:36.282948 9435 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59946:
table { table_name: "dugtrio" } modify_external_catalogs: true
I20251209 03:05:36.283921 9435 catalog_manager.cc:5941] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: Sending DeleteTablet for 3 replicas of tablet 09cd21d9bc7a4eb199bf120c0892a077
I20251209 03:05:36.284029 9435 catalog_manager.cc:5941] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: Sending DeleteTablet for 3 replicas of tablet acfa3c3152b84ba1973e14357cd67fc6
I20251209 03:05:36.284067 9435 catalog_manager.cc:5941] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: Sending DeleteTablet for 3 replicas of tablet 9de8c0db7da244ba8bdeecbc3350bba2
I20251209 03:05:36.284103 9435 catalog_manager.cc:5941] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: Sending DeleteTablet for 3 replicas of tablet cac190172b0144b3a3c3b9e1de80dcab
I20251209 03:05:36.284313 9681 tablet_service.cc:1552] Processing DeleteTablet for tablet 09cd21d9bc7a4eb199bf120c0892a077 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:49650
I20251209 03:05:36.284346 9679 tablet_service.cc:1552] Processing DeleteTablet for tablet 9de8c0db7da244ba8bdeecbc3350bba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:49650
I20251209 03:05:36.284438 9606 tablet_service.cc:1552] Processing DeleteTablet for tablet 09cd21d9bc7a4eb199bf120c0892a077 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:34546
I20251209 03:05:36.284457 9605 tablet_service.cc:1552] Processing DeleteTablet for tablet acfa3c3152b84ba1973e14357cd67fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:34546
I20251209 03:05:36.284510 9603 tablet_service.cc:1552] Processing DeleteTablet for tablet cac190172b0144b3a3c3b9e1de80dcab with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:34546
I20251209 03:05:36.284571 9604 tablet_service.cc:1552] Processing DeleteTablet for tablet 9de8c0db7da244ba8bdeecbc3350bba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:34546
I20251209 03:05:36.284583 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:05:36.284363 9680 tablet_service.cc:1552] Processing DeleteTablet for tablet cac190172b0144b3a3c3b9e1de80dcab with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:49650
I20251209 03:05:36.284652 9822 tablet_replica.cc:333] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: stopping tablet replica
I20251209 03:05:36.284346 9675 tablet_service.cc:1552] Processing DeleteTablet for tablet acfa3c3152b84ba1973e14357cd67fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:49650
I20251209 03:05:36.284605 9823 tablet_replica.cc:333] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: stopping tablet replica
I20251209 03:05:36.284701 9822 raft_consensus.cc:2243] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:36.284727 9823 raft_consensus.cc:2243] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:36.284740 9822 raft_consensus.cc:2272] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.284791 9823 raft_consensus.cc:2272] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.285022 9823 ts_tablet_manager.cc:1916] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251209 03:05:36.285005 9424 catalog_manager.cc:4712] TS aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cac190172b0144b3a3c3b9e1de80dcab: Network error: Client connection negotiation failed: client connection to 127.2.170.193:43401: connect: Connection refused (error 111)
I20251209 03:05:36.285125 9822 ts_tablet_manager.cc:1916] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251209 03:05:36.286655 9825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:36.286736 9826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:36.286875 9823 ts_tablet_manager.cc:1929] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251209 03:05:36.286899 2731 server_base.cc:1047] running on GCE node
I20251209 03:05:36.286980 9823 log.cc:1199] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/09cd21d9bc7a4eb199bf120c0892a077
I20251209 03:05:36.287201 9823 ts_tablet_manager.cc:1950] T 09cd21d9bc7a4eb199bf120c0892a077 P f106240450834d39a486b36a4ea76851: Deleting consensus metadata
I20251209 03:05:36.287442 9823 tablet_replica.cc:333] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: stopping tablet replica
I20251209 03:05:36.287492 9822 ts_tablet_manager.cc:1929] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251209 03:05:36.287511 9823 raft_consensus.cc:2243] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:36.287523 9421 catalog_manager.cc:4985] TS f106240450834d39a486b36a4ea76851 (127.2.170.194:38749): tablet 09cd21d9bc7a4eb199bf120c0892a077 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.287547 9822 log.cc:1199] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/09cd21d9bc7a4eb199bf120c0892a077
I20251209 03:05:36.287547 9823 raft_consensus.cc:2272] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.287774 9822 ts_tablet_manager.cc:1950] T 09cd21d9bc7a4eb199bf120c0892a077 P 5a68e1323b54428b9a4b841a924829cb: Deleting consensus metadata
I20251209 03:05:36.287783 9823 ts_tablet_manager.cc:1916] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251209 03:05:36.287968 9828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:36.288203 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:36.288257 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:36.288283 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249536288284 us; error 0 us; skew 500 ppm
I20251209 03:05:36.288215 9822 tablet_replica.cc:333] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: stopping tablet replica
I20251209 03:05:36.288391 9822 raft_consensus.cc:2243] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:36.288465 9822 raft_consensus.cc:2272] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.288262 9422 catalog_manager.cc:4985] TS 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713): tablet 09cd21d9bc7a4eb199bf120c0892a077 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.288806 9822 ts_tablet_manager.cc:1916] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.289001 9823 ts_tablet_manager.cc:1929] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251209 03:05:36.289048 9823 log.cc:1199] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/acfa3c3152b84ba1973e14357cd67fc6
I20251209 03:05:36.289142 2731 webserver.cc:492] Webserver started at http://127.2.170.196:41311/ using document root <none> and password file <none>
I20251209 03:05:36.289218 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:36.289232 9823 ts_tablet_manager.cc:1950] T acfa3c3152b84ba1973e14357cd67fc6 P f106240450834d39a486b36a4ea76851: Deleting consensus metadata
I20251209 03:05:36.289258 2731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251209 03:05:36.289299 2731 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251209 03:05:36.289541 9823 tablet_replica.cc:333] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: stopping tablet replica
I20251209 03:05:36.289573 2731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-3-root/instance:
uuid: "60069bea3a4d425d9176c6c96143e2c4"
format_stamp: "Formatted at 2025-12-09 03:05:36 on dist-test-slave-4cb0"
I20251209 03:05:36.289589 9823 raft_consensus.cc:2243] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:36.289611 9421 catalog_manager.cc:4985] TS f106240450834d39a486b36a4ea76851 (127.2.170.194:38749): tablet acfa3c3152b84ba1973e14357cd67fc6 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.289664 9823 raft_consensus.cc:2272] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.289923 9823 ts_tablet_manager.cc:1916] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.290087 9822 ts_tablet_manager.cc:1929] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251209 03:05:36.290134 9822 log.cc:1199] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/9de8c0db7da244ba8bdeecbc3350bba2
I20251209 03:05:36.290311 9822 ts_tablet_manager.cc:1950] T 9de8c0db7da244ba8bdeecbc3350bba2 P 5a68e1323b54428b9a4b841a924829cb: Deleting consensus metadata
I20251209 03:05:36.290596 9822 tablet_replica.cc:333] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: stopping tablet replica
I20251209 03:05:36.290601 9422 catalog_manager.cc:4985] TS 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713): tablet 9de8c0db7da244ba8bdeecbc3350bba2 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.290632 2731 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20251209 03:05:36.290653 9822 raft_consensus.cc:2243] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:36.290720 9822 raft_consensus.cc:2272] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.290905 9822 ts_tablet_manager.cc:1916] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.291181 9833 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.291190 9823 ts_tablet_manager.cc:1929] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251209 03:05:36.291265 9823 log.cc:1199] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/cac190172b0144b3a3c3b9e1de80dcab
I20251209 03:05:36.291343 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.291389 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-3-root
uuid: "60069bea3a4d425d9176c6c96143e2c4"
format_stamp: "Formatted at 2025-12-09 03:05:36 on dist-test-slave-4cb0"
I20251209 03:05:36.291432 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:36.291453 9823 ts_tablet_manager.cc:1950] T cac190172b0144b3a3c3b9e1de80dcab P f106240450834d39a486b36a4ea76851: Deleting consensus metadata
I20251209 03:05:36.291636 9823 tablet_replica.cc:333] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: stopping tablet replica
I20251209 03:05:36.291680 9823 raft_consensus.cc:2243] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:36.291708 9823 raft_consensus.cc:2272] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.291720 9421 catalog_manager.cc:4985] TS f106240450834d39a486b36a4ea76851 (127.2.170.194:38749): tablet cac190172b0144b3a3c3b9e1de80dcab (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.291929 9823 ts_tablet_manager.cc:1916] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.292050 9822 ts_tablet_manager.cc:1929] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251209 03:05:36.292100 9822 log.cc:1199] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/cac190172b0144b3a3c3b9e1de80dcab
I20251209 03:05:36.292274 9822 ts_tablet_manager.cc:1950] T cac190172b0144b3a3c3b9e1de80dcab P 5a68e1323b54428b9a4b841a924829cb: Deleting consensus metadata
I20251209 03:05:36.292574 9422 catalog_manager.cc:4985] TS 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713): tablet cac190172b0144b3a3c3b9e1de80dcab (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.292657 9822 tablet_replica.cc:333] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: stopping tablet replica
I20251209 03:05:36.292697 9822 raft_consensus.cc:2243] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:36.292749 9822 raft_consensus.cc:2272] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.292968 9823 ts_tablet_manager.cc:1929] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251209 03:05:36.293028 9823 log.cc:1199] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/9de8c0db7da244ba8bdeecbc3350bba2
I20251209 03:05:36.293027 9822 ts_tablet_manager.cc:1916] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.293254 9823 ts_tablet_manager.cc:1950] T 9de8c0db7da244ba8bdeecbc3350bba2 P f106240450834d39a486b36a4ea76851: Deleting consensus metadata
I20251209 03:05:36.293503 9421 catalog_manager.cc:4985] TS f106240450834d39a486b36a4ea76851 (127.2.170.194:38749): tablet 9de8c0db7da244ba8bdeecbc3350bba2 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.294126 9822 ts_tablet_manager.cc:1929] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251209 03:05:36.294180 9822 log.cc:1199] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/acfa3c3152b84ba1973e14357cd67fc6
I20251209 03:05:36.294342 9822 ts_tablet_manager.cc:1950] T acfa3c3152b84ba1973e14357cd67fc6 P 5a68e1323b54428b9a4b841a924829cb: Deleting consensus metadata
I20251209 03:05:36.294757 9422 catalog_manager.cc:4985] TS 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713): tablet acfa3c3152b84ba1973e14357cd67fc6 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.296435 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:36.296579 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:36.296667 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:36.296782 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251209 03:05:36.296803 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.296823 2731 ts_tablet_manager.cc:616] Registered 0 tablets
I20251209 03:05:36.296838 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.299329 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.196:44759
I20251209 03:05:36.299818 2731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251209 03:05:36.299940 9896 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.196:44759 every 8 connection(s)
W20251209 03:05:36.300915 9899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251209 03:05:36.300981 9900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:36.301160 2731 server_base.cc:1047] running on GCE node
W20251209 03:05:36.301609 9902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251209 03:05:36.301777 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251209 03:05:36.301813 2731 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251209 03:05:36.301831 2731 hybrid_clock.cc:648] HybridClock initialized: now 1765249536301830 us; error 0 us; skew 500 ppm
I20251209 03:05:36.302489 2731 webserver.cc:492] Webserver started at http://127.2.170.193:38939/ using document root <none> and password file <none>
I20251209 03:05:36.302572 2731 fs_manager.cc:362] Metadata directory not provided
I20251209 03:05:36.302597 2731 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251209 03:05:36.303117 2731 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.303162 9897 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41961
I20251209 03:05:36.303228 9897 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:36.303333 9897 heartbeater.cc:507] Master 127.2.170.254:41961 requested a full tablet report, sending...
I20251209 03:05:36.303565 9435 ts_manager.cc:194] Registered new tserver with Master: 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759)
I20251209 03:05:36.303887 9908 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.304006 2731 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.304004 9435 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59974
I20251209 03:05:36.304070 2731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root
uuid: "aa74703951b844639caf72b3bc20d475"
format_stamp: "Formatted at 2025-12-09 03:05:33 on dist-test-slave-4cb0"
I20251209 03:05:36.304111 2731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251209 03:05:36.320919 2731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251209 03:05:36.321106 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251209 03:05:36.321240 2731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251209 03:05:36.321525 9915 ts_tablet_manager.cc:542] Loading tablet metadata (0/8 complete)
I20251209 03:05:36.323549 2731 ts_tablet_manager.cc:585] Loaded tablet metadata (8 total tablets, 8 live tablets)
I20251209 03:05:36.323593 2731 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20251209 03:05:36.323624 2731 ts_tablet_manager.cc:600] Registering tablets (0/8 complete)
I20251209 03:05:36.324045 9915 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:36.325002 9915 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:36.325194 9915 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Bootstrap complete.
I20251209 03:05:36.325212 2731 ts_tablet_manager.cc:616] Registered 8 tablets
I20251209 03:05:36.325244 2731 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20251209 03:05:36.325279 9915 ts_tablet_manager.cc:1403] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:36.325399 9915 raft_consensus.cc:359] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.325455 9915 raft_consensus.cc:740] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:36.325495 9915 consensus_queue.cc:260] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.325644 9915 ts_tablet_manager.cc:1434] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:36.325721 9915 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:36.326588 9915 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:36.326730 9915 tablet_bootstrap.cc:492] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Bootstrap complete.
I20251209 03:05:36.326824 9915 ts_tablet_manager.cc:1403] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:36.326936 9915 raft_consensus.cc:359] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.327003 9915 raft_consensus.cc:740] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:36.327046 9915 consensus_queue.cc:260] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.327124 9915 ts_tablet_manager.cc:1434] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.327174 9915 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:36.327852 9915 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:36.327983 9915 tablet_bootstrap.cc:492] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Bootstrap complete.
I20251209 03:05:36.328094 9915 ts_tablet_manager.cc:1403] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20251209 03:05:36.328207 9915 raft_consensus.cc:359] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.328253 9915 raft_consensus.cc:740] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:36.328294 9915 consensus_queue.cc:260] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.328372 9915 ts_tablet_manager.cc:1434] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.328431 9915 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:36.328528 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.170.193:43401
I20251209 03:05:36.328576 9976 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.170.193:43401 every 8 connection(s)
I20251209 03:05:36.329341 9915 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:36.329514 9915 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Bootstrap complete.
I20251209 03:05:36.329595 9915 ts_tablet_manager.cc:1403] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:36.329741 9915 raft_consensus.cc:359] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.329789 9915 raft_consensus.cc:740] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:36.329826 9915 consensus_queue.cc:260] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.329908 9915 ts_tablet_manager.cc:1434] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:36.329957 9915 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:36.330885 9915 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:36.331050 9915 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Bootstrap complete.
I20251209 03:05:36.331137 9915 ts_tablet_manager.cc:1403] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:36.331262 9915 raft_consensus.cc:359] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.331308 9915 raft_consensus.cc:740] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:36.331364 9915 consensus_queue.cc:260] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.331457 9915 ts_tablet_manager.cc:1434] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:36.331514 9915 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:36.331722 9977 heartbeater.cc:344] Connected to a master server at 127.2.170.254:41961
I20251209 03:05:36.331779 9977 heartbeater.cc:461] Registering TS with master...
I20251209 03:05:36.331894 9977 heartbeater.cc:507] Master 127.2.170.254:41961 requested a full tablet report, sending...
I20251209 03:05:36.332191 9435 ts_manager.cc:194] Re-registered known tserver with Master: aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401)
I20251209 03:05:36.332568 9915 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:36.332583 9435 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59990
I20251209 03:05:36.332751 9915 tablet_bootstrap.cc:492] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Bootstrap complete.
I20251209 03:05:36.332842 9915 ts_tablet_manager.cc:1403] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:36.332962 9915 raft_consensus.cc:359] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.332999 9977 heartbeater.cc:499] Master 127.2.170.254:41961 was elected leader, sending a full tablet report...
I20251209 03:05:36.333006 9915 raft_consensus.cc:740] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:36.333074 9915 consensus_queue.cc:260] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.333181 9915 ts_tablet_manager.cc:1434] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:36.333231 9915 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:36.334087 9915 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:36.334216 9915 tablet_bootstrap.cc:492] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Bootstrap complete.
I20251209 03:05:36.334297 9915 ts_tablet_manager.cc:1403] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:36.334441 9915 raft_consensus.cc:359] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.334491 9915 raft_consensus.cc:740] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:36.334537 9915 consensus_queue.cc:260] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.334622 9915 ts_tablet_manager.cc:1434] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.334674 9915 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Bootstrap starting.
I20251209 03:05:36.335525 9915 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:36.335633 9915 tablet_bootstrap.cc:492] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Bootstrap complete.
I20251209 03:05:36.335695 9915 ts_tablet_manager.cc:1403] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:36.335813 9915 raft_consensus.cc:359] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.335870 9915 raft_consensus.cc:740] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa74703951b844639caf72b3bc20d475, State: Initialized, Role: FOLLOWER
I20251209 03:05:36.335937 9915 consensus_queue.cc:260] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.336019 9915 ts_tablet_manager.cc:1434] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:36.338353 9940 tablet_service.cc:1552] Processing DeleteTablet for tablet 9de8c0db7da244ba8bdeecbc3350bba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:40184
I20251209 03:05:36.338531 9986 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:36.338587 9986 raft_consensus.cc:2243] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:36.338625 9986 raft_consensus.cc:2272] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.338785 9986 ts_tablet_manager.cc:1916] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.340152 9986 ts_tablet_manager.cc:1929] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251209 03:05:36.340199 9986 log.cc:1199] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/9de8c0db7da244ba8bdeecbc3350bba2
I20251209 03:05:36.340348 9986 ts_tablet_manager.cc:1950] T 9de8c0db7da244ba8bdeecbc3350bba2 P aa74703951b844639caf72b3bc20d475: Deleting consensus metadata
I20251209 03:05:36.340653 9424 catalog_manager.cc:4985] TS aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): tablet 9de8c0db7da244ba8bdeecbc3350bba2 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.340818 9940 tablet_service.cc:1552] Processing DeleteTablet for tablet acfa3c3152b84ba1973e14357cd67fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:40184
I20251209 03:05:36.340924 9986 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:36.340988 9986 raft_consensus.cc:2243] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:36.341027 9986 raft_consensus.cc:2272] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.341290 9986 ts_tablet_manager.cc:1916] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.342507 9986 ts_tablet_manager.cc:1929] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251209 03:05:36.342558 9986 log.cc:1199] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/acfa3c3152b84ba1973e14357cd67fc6
I20251209 03:05:36.342689 9986 ts_tablet_manager.cc:1950] T acfa3c3152b84ba1973e14357cd67fc6 P aa74703951b844639caf72b3bc20d475: Deleting consensus metadata
I20251209 03:05:36.342945 9424 catalog_manager.cc:4985] TS aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): tablet acfa3c3152b84ba1973e14357cd67fc6 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.346230 9940 tablet_service.cc:1552] Processing DeleteTablet for tablet cac190172b0144b3a3c3b9e1de80dcab with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:40184
I20251209 03:05:36.346323 9986 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:36.346396 9986 raft_consensus.cc:2243] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:36.346439 9986 raft_consensus.cc:2272] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.346616 9986 ts_tablet_manager.cc:1916] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.347932 9986 ts_tablet_manager.cc:1929] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251209 03:05:36.347997 9986 log.cc:1199] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/cac190172b0144b3a3c3b9e1de80dcab
I20251209 03:05:36.348271 9986 ts_tablet_manager.cc:1950] T cac190172b0144b3a3c3b9e1de80dcab P aa74703951b844639caf72b3bc20d475: Deleting consensus metadata
I20251209 03:05:36.348631 9424 catalog_manager.cc:4985] TS aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): tablet cac190172b0144b3a3c3b9e1de80dcab (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.348809 9940 tablet_service.cc:1552] Processing DeleteTablet for tablet 09cd21d9bc7a4eb199bf120c0892a077 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-09 03:05:36 UTC) from {username='slave'} at 127.0.0.1:40184
I20251209 03:05:36.348892 9986 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:36.348953 9986 raft_consensus.cc:2243] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:36.348991 9986 raft_consensus.cc:2272] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:36.349195 9986 ts_tablet_manager.cc:1916] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251209 03:05:36.350229 9986 ts_tablet_manager.cc:1929] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251209 03:05:36.350278 9986 log.cc:1199] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/09cd21d9bc7a4eb199bf120c0892a077
I20251209 03:05:36.350430 9986 ts_tablet_manager.cc:1950] T 09cd21d9bc7a4eb199bf120c0892a077 P aa74703951b844639caf72b3bc20d475: Deleting consensus metadata
I20251209 03:05:36.350667 9691 consensus_queue.cc:237] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:36.350708 9424 catalog_manager.cc:4985] TS aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): tablet 09cd21d9bc7a4eb199bf120c0892a077 (table dugtrio [id=4869122f1d3645f09c04569da8a692a2]) successfully deleted
I20251209 03:05:36.351222 9616 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:05:36.351397 9745 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:36.352435 9804 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:36.353283 9422 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4135fcbfa27d4b438a22b2319f49bbd4 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 4)
I20251209 03:05:36.353345 9435 catalog_manager.cc:5654] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb reported cstate change: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:36.353425 9616 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:36.354293 9950 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:05:36.354633 9757 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:36.355041 9950 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
W20251209 03:05:36.355289 9655 consensus_peers.cc:597] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb -> Peer 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): Couldn't send request to peer 60069bea3a4d425d9176c6c96143e2c4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4135fcbfa27d4b438a22b2319f49bbd4. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:36.395610 9745 consensus_queue.cc:579] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Leader has been unable to successfully communicate with peer aa74703951b844639caf72b3bc20d475 for more than 1 seconds (1.143s)
I20251209 03:05:36.396462 9691 consensus_queue.cc:237] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:36.396992 9616 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:05:36.396999 9950 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251209 03:05:36.397082 9655 consensus_peers.cc:597] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb -> Peer 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): Couldn't send request to peer 60069bea3a4d425d9176c6c96143e2c4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2c804e735df449eab77a465a518f6ff. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:36.397195 9757 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:36.397286 9745 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:36.398126 9745 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:36.398270 9616 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:36.399111 9950 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:36.399171 9422 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet b2c804e735df449eab77a465a518f6ff with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251209 03:05:36.399191 9436 catalog_manager.cc:5654] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 reported cstate change: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:36.400030 9691 consensus_queue.cc:237] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.400403 9950 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:36.400403 9616 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:36.400599 9757 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:36.400707 9804 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:36.401100 9757 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.401219 9616 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.401962 9422 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet b2c804e735df449eab77a465a518f6ff with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:36.401940 9950 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.402078 9435 catalog_manager.cc:5654] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 reported cstate change: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.405236 9861 tablet_service.cc:1552] Processing DeleteTablet for tablet b2c804e735df449eab77a465a518f6ff with delete_type TABLET_DATA_TOMBSTONED (TS 60069bea3a4d425d9176c6c96143e2c4 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44986
W20251209 03:05:36.405635 9422 catalog_manager.cc:4952] TS 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): delete failed for tablet b2c804e735df449eab77a465a518f6ff because the tablet was not found. No further retry: Not found: Tablet not found: b2c804e735df449eab77a465a518f6ff
I20251209 03:05:36.419683 9616 consensus_queue.cc:237] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:36.420460 9691 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:05:36.420681 9998 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:36.421519 9997 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:36.422720 9421 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 2882613655374b0ab2fedbfb0676349d with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 6)
I20251209 03:05:36.422920 9691 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:36.422971 9435 catalog_manager.cc:5654] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 reported cstate change: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "f106240450834d39a486b36a4ea76851" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251209 03:05:36.423022 9950 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251209 03:05:36.423466 9998 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:36.423870 9950 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
W20251209 03:05:36.423969 9580 consensus_peers.cc:597] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 -> Peer 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): Couldn't send request to peer 60069bea3a4d425d9176c6c96143e2c4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2882613655374b0ab2fedbfb0676349d. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:36.646790 9804 raft_consensus.cc:493] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa74703951b844639caf72b3bc20d475)
I20251209 03:05:36.646904 9804 raft_consensus.cc:515] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.647053 9804 leader_election.cc:290] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:36.647282 9950 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475" is_pre_election: true
I20251209 03:05:36.647285 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851" is_pre_election: true
I20251209 03:05:36.647387 9950 raft_consensus.cc:2468] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:36.647411 9616 raft_consensus.cc:2468] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 1.
I20251209 03:05:36.647526 9657 leader_election.cc:304] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, aa74703951b844639caf72b3bc20d475; no voters:
I20251209 03:05:36.647625 9804 raft_consensus.cc:2804] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251209 03:05:36.647656 9804 raft_consensus.cc:493] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa74703951b844639caf72b3bc20d475)
I20251209 03:05:36.647675 9804 raft_consensus.cc:3060] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:36.648360 9804 raft_consensus.cc:515] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.648490 9804 leader_election.cc:290] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 2 election: Requested vote from peers aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401), f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:36.648619 9616 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f106240450834d39a486b36a4ea76851"
I20251209 03:05:36.648640 9950 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed91b26a88e54442888350505b700d79" candidate_uuid: "5a68e1323b54428b9a4b841a924829cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa74703951b844639caf72b3bc20d475"
I20251209 03:05:36.648681 9616 raft_consensus.cc:3060] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:36.648684 9950 raft_consensus.cc:3060] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Advancing to term 2
I20251209 03:05:36.649308 9616 raft_consensus.cc:2468] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 2.
I20251209 03:05:36.649308 9950 raft_consensus.cc:2468] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a68e1323b54428b9a4b841a924829cb in term 2.
I20251209 03:05:36.649502 9654 leader_election.cc:304] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a68e1323b54428b9a4b841a924829cb, f106240450834d39a486b36a4ea76851; no voters:
I20251209 03:05:36.649592 9804 raft_consensus.cc:2804] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 2 FOLLOWER]: Leader election won for term 2
I20251209 03:05:36.649636 9804 raft_consensus.cc:697] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 2 LEADER]: Becoming Leader. State: Replica: 5a68e1323b54428b9a4b841a924829cb, State: Running, Role: LEADER
I20251209 03:05:36.649700 9804 consensus_queue.cc:237] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.650254 9435 catalog_manager.cc:5654] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb reported cstate change: term changed from 1 to 2, leader changed from aa74703951b844639caf72b3bc20d475 (127.2.170.193) to 5a68e1323b54428b9a4b841a924829cb (127.2.170.195). New cstate: current_term: 2 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } }
I20251209 03:05:36.812569 9691 consensus_queue.cc:237] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.813020 9616 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:36.812995 9950 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:36.813249 9757 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:36.813320 9804 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:36.813970 9804 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.814098 9616 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.814158 9950 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.815017 9422 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 4135fcbfa27d4b438a22b2319f49bbd4 with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:36.815191 9435 catalog_manager.cc:5654] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 reported cstate change: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.815953 9861 tablet_service.cc:1552] Processing DeleteTablet for tablet 4135fcbfa27d4b438a22b2319f49bbd4 with delete_type TABLET_DATA_TOMBSTONED (TS 60069bea3a4d425d9176c6c96143e2c4 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44986
W20251209 03:05:36.816160 9422 catalog_manager.cc:4952] TS 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): delete failed for tablet 4135fcbfa27d4b438a22b2319f49bbd4 because the tablet was not found. No further retry: Not found: Tablet not found: 4135fcbfa27d4b438a22b2319f49bbd4
I20251209 03:05:36.872800 9616 consensus_queue.cc:237] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } }
I20251209 03:05:36.873214 9950 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:36.873315 9691 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251209 03:05:36.873390 9997 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:36.873505 9997 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251209 03:05:36.874029 9998 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.874167 9950 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.874198 9691 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.874953 9421 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 2882613655374b0ab2fedbfb0676349d with cas_config_opid_index 2: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:36.875227 9437 catalog_manager.cc:5654] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 reported cstate change: config changed from index 2 to 3, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) evicted. New cstate: current_term: 1 leader_uuid: "f106240450834d39a486b36a4ea76851" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } }
I20251209 03:05:36.875977 9861 tablet_service.cc:1552] Processing DeleteTablet for tablet 2882613655374b0ab2fedbfb0676349d with delete_type TABLET_DATA_TOMBSTONED (TS 60069bea3a4d425d9176c6c96143e2c4 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44986
W20251209 03:05:36.876163 9422 catalog_manager.cc:4952] TS 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): delete failed for tablet 2882613655374b0ab2fedbfb0676349d because the tablet was not found. No further retry: Not found: Tablet not found: 2882613655374b0ab2fedbfb0676349d
I20251209 03:05:37.043742 9950 raft_consensus.cc:1275] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 2 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251209 03:05:37.044013 9804 consensus_queue.cc:1048] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:37.045137 9616 raft_consensus.cc:1275] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 2 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251209 03:05:37.045573 9804 consensus_queue.cc:1048] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251209 03:05:37.094323 9616 consensus_queue.cc:237] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.094767 9691 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:37.094795 9950 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20251209 03:05:37.094937 9580 consensus_peers.cc:597] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 -> Peer 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): Couldn't send request to peer 60069bea3a4d425d9176c6c96143e2c4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2882613655374b0ab2fedbfb0676349d. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:37.094964 9998 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:37.095063 9814 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:37.095621 9997 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.095712 9691 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.095755 9950 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.096640 9435 catalog_manager.cc:5654] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb reported cstate change: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "f106240450834d39a486b36a4ea76851" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.099378 9691 consensus_queue.cc:237] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.099795 9616 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:37.099891 9950 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20251209 03:05:37.099910 9655 consensus_peers.cc:597] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb -> Peer 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): Couldn't send request to peer 60069bea3a4d425d9176c6c96143e2c4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4135fcbfa27d4b438a22b2319f49bbd4. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:37.099951 9757 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:37.100041 9804 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:37.100538 9745 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.100653 9950 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.100658 9616 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.101815 9437 catalog_manager.cc:5654] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 reported cstate change: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.101964 9691 consensus_queue.cc:237] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.102298 9616 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:37.102424 9950 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20251209 03:05:37.102475 9745 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251209 03:05:37.102582 9804 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20251209 03:05:37.102646 9655 consensus_peers.cc:597] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb -> Peer 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196:44759): Couldn't send request to peer 60069bea3a4d425d9176c6c96143e2c4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2c804e735df449eab77a465a518f6ff. This is attempt 1: this message will repeat every 5th retry.
I20251209 03:05:37.103044 9745 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.103224 9616 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.103220 9950 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.104091 9435 catalog_manager.cc:5654] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 reported cstate change: config changed from index 3 to 4, NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) added. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } }
I20251209 03:05:37.304497 9897 heartbeater.cc:499] Master 127.2.170.254:41961 was elected leader, sending a full tablet report...
I20251209 03:05:37.655959 10011 ts_tablet_manager.cc:933] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: Initiating tablet copy from peer 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:37.656279 10011 tablet_copy_client.cc:323] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.195:44713
I20251209 03:05:37.659554 9701 tablet_copy_service.cc:140] P 5a68e1323b54428b9a4b841a924829cb: Received BeginTabletCopySession request for tablet b2c804e735df449eab77a465a518f6ff from peer 60069bea3a4d425d9176c6c96143e2c4 ({username='slave'} at 127.0.0.1:49670)
I20251209 03:05:37.659632 9701 tablet_copy_service.cc:161] P 5a68e1323b54428b9a4b841a924829cb: Beginning new tablet copy session on tablet b2c804e735df449eab77a465a518f6ff from peer 60069bea3a4d425d9176c6c96143e2c4 at {username='slave'} at 127.0.0.1:49670: session id = 60069bea3a4d425d9176c6c96143e2c4-b2c804e735df449eab77a465a518f6ff
I20251209 03:05:37.660195 9701 tablet_copy_source_session.cc:215] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:05:37.660540 10011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2c804e735df449eab77a465a518f6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:37.661552 10011 tablet_copy_client.cc:806] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Starting download of 0 data blocks...
I20251209 03:05:37.661695 10011 tablet_copy_client.cc:670] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Starting download of 1 WAL segments...
I20251209 03:05:37.662215 10011 tablet_copy_client.cc:538] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:05:37.662822 10011 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap starting.
I20251209 03:05:37.664166 10011 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:37.664376 10011 tablet_bootstrap.cc:492] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap complete.
I20251209 03:05:37.664474 10011 ts_tablet_manager.cc:1403] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251209 03:05:37.664634 10011 raft_consensus.cc:359] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.664698 10011 raft_consensus.cc:740] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 60069bea3a4d425d9176c6c96143e2c4, State: Initialized, Role: LEARNER
I20251209 03:05:37.664726 10011 consensus_queue.cc:260] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.664898 10011 ts_tablet_manager.cc:1434] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20251209 03:05:37.665081 9701 tablet_copy_service.cc:342] P 5a68e1323b54428b9a4b841a924829cb: Request end of tablet copy session 60069bea3a4d425d9176c6c96143e2c4-b2c804e735df449eab77a465a518f6ff received from {username='slave'} at 127.0.0.1:49670
I20251209 03:05:37.665129 9701 tablet_copy_service.cc:434] P 5a68e1323b54428b9a4b841a924829cb: ending tablet copy session 60069bea3a4d425d9176c6c96143e2c4-b2c804e735df449eab77a465a518f6ff on tablet b2c804e735df449eab77a465a518f6ff with peer 60069bea3a4d425d9176c6c96143e2c4
I20251209 03:05:37.671072 10011 ts_tablet_manager.cc:933] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: Initiating tablet copy from peer 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713)
I20251209 03:05:37.671269 10011 tablet_copy_client.cc:323] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.195:44713
I20251209 03:05:37.671469 9701 tablet_copy_service.cc:140] P 5a68e1323b54428b9a4b841a924829cb: Received BeginTabletCopySession request for tablet 4135fcbfa27d4b438a22b2319f49bbd4 from peer 60069bea3a4d425d9176c6c96143e2c4 ({username='slave'} at 127.0.0.1:49670)
I20251209 03:05:37.671535 9701 tablet_copy_service.cc:161] P 5a68e1323b54428b9a4b841a924829cb: Beginning new tablet copy session on tablet 4135fcbfa27d4b438a22b2319f49bbd4 from peer 60069bea3a4d425d9176c6c96143e2c4 at {username='slave'} at 127.0.0.1:49670: session id = 60069bea3a4d425d9176c6c96143e2c4-4135fcbfa27d4b438a22b2319f49bbd4
I20251209 03:05:37.672119 9701 tablet_copy_source_session.cc:215] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:05:37.672422 10011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4135fcbfa27d4b438a22b2319f49bbd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:37.673457 10011 tablet_copy_client.cc:806] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Starting download of 0 data blocks...
I20251209 03:05:37.673564 10011 tablet_copy_client.cc:670] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Starting download of 1 WAL segments...
I20251209 03:05:37.674058 10011 tablet_copy_client.cc:538] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:05:37.674598 10011 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap starting.
I20251209 03:05:37.675729 10011 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:37.675873 10011 tablet_bootstrap.cc:492] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap complete.
I20251209 03:05:37.675961 10011 ts_tablet_manager.cc:1403] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20251209 03:05:37.676100 10011 raft_consensus.cc:359] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.676162 10011 raft_consensus.cc:740] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 60069bea3a4d425d9176c6c96143e2c4, State: Initialized, Role: LEARNER
I20251209 03:05:37.676192 10011 consensus_queue.cc:260] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.676280 10011 ts_tablet_manager.cc:1434] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:37.676503 9701 tablet_copy_service.cc:342] P 5a68e1323b54428b9a4b841a924829cb: Request end of tablet copy session 60069bea3a4d425d9176c6c96143e2c4-4135fcbfa27d4b438a22b2319f49bbd4 received from {username='slave'} at 127.0.0.1:49670
I20251209 03:05:37.676566 9701 tablet_copy_service.cc:434] P 5a68e1323b54428b9a4b841a924829cb: ending tablet copy session 60069bea3a4d425d9176c6c96143e2c4-4135fcbfa27d4b438a22b2319f49bbd4 on tablet 4135fcbfa27d4b438a22b2319f49bbd4 with peer 60069bea3a4d425d9176c6c96143e2c4
I20251209 03:05:37.681923 10011 ts_tablet_manager.cc:933] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: Initiating tablet copy from peer f106240450834d39a486b36a4ea76851 (127.2.170.194:38749)
I20251209 03:05:37.682078 10011 tablet_copy_client.cc:323] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Beginning tablet copy session from remote peer at address 127.2.170.194:38749
I20251209 03:05:37.685101 9626 tablet_copy_service.cc:140] P f106240450834d39a486b36a4ea76851: Received BeginTabletCopySession request for tablet 2882613655374b0ab2fedbfb0676349d from peer 60069bea3a4d425d9176c6c96143e2c4 ({username='slave'} at 127.0.0.1:34592)
I20251209 03:05:37.685173 9626 tablet_copy_service.cc:161] P f106240450834d39a486b36a4ea76851: Beginning new tablet copy session on tablet 2882613655374b0ab2fedbfb0676349d from peer 60069bea3a4d425d9176c6c96143e2c4 at {username='slave'} at 127.0.0.1:34592: session id = 60069bea3a4d425d9176c6c96143e2c4-2882613655374b0ab2fedbfb0676349d
I20251209 03:05:37.685604 9626 tablet_copy_source_session.cc:215] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851: Tablet Copy: opened 0 blocks and 1 log segments
I20251209 03:05:37.685925 10011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2882613655374b0ab2fedbfb0676349d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251209 03:05:37.686962 10011 tablet_copy_client.cc:806] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Starting download of 0 data blocks...
I20251209 03:05:37.687072 10011 tablet_copy_client.cc:670] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Starting download of 1 WAL segments...
I20251209 03:05:37.687534 10011 tablet_copy_client.cc:538] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251209 03:05:37.688133 10011 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap starting.
I20251209 03:05:37.689517 10011 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251209 03:05:37.689718 10011 tablet_bootstrap.cc:492] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: Bootstrap complete.
I20251209 03:05:37.689829 10011 ts_tablet_manager.cc:1403] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20251209 03:05:37.689994 10011 raft_consensus.cc:359] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.690071 10011 raft_consensus.cc:740] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 60069bea3a4d425d9176c6c96143e2c4, State: Initialized, Role: LEARNER
I20251209 03:05:37.690105 10011 consensus_queue.cc:260] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } }
I20251209 03:05:37.690183 10011 ts_tablet_manager.cc:1434] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20251209 03:05:37.690397 9626 tablet_copy_service.cc:342] P f106240450834d39a486b36a4ea76851: Request end of tablet copy session 60069bea3a4d425d9176c6c96143e2c4-2882613655374b0ab2fedbfb0676349d received from {username='slave'} at 127.0.0.1:34592
I20251209 03:05:37.690443 9626 tablet_copy_service.cc:434] P f106240450834d39a486b36a4ea76851: ending tablet copy session 60069bea3a4d425d9176c6c96143e2c4-2882613655374b0ab2fedbfb0676349d on tablet 2882613655374b0ab2fedbfb0676349d with peer 60069bea3a4d425d9176c6c96143e2c4
I20251209 03:05:38.135494 9871 raft_consensus.cc:1217] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251209 03:05:38.167655 9871 raft_consensus.cc:1217] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251209 03:05:38.186007 9871 raft_consensus.cc:1217] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20251209 03:05:38.579155 9997 raft_consensus.cc:1064] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851: attempting to promote NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 to VOTER
I20251209 03:05:38.579381 9997 consensus_queue.cc:237] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } }
I20251209 03:05:38.579875 9950 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.579885 9691 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.579885 9871 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.580132 10024 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.580268 10027 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.580518 9997 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.581209 10027 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.581352 9950 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.581724 9871 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.582669 9691 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.582712 9436 catalog_manager.cc:5654] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 reported cstate change: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f106240450834d39a486b36a4ea76851" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:05:38.584692 9615 consensus_queue.cc:237] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } }
I20251209 03:05:38.585131 9871 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251209 03:05:38.585160 9950 raft_consensus.cc:1275] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer f106240450834d39a486b36a4ea76851: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251209 03:05:38.585395 10027 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251209 03:05:38.585654 10027 consensus_queue.cc:1048] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251209 03:05:38.586035 10026 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5a68e1323b54428b9a4b841a924829cb (127.2.170.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.586295 9871 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5a68e1323b54428b9a4b841a924829cb (127.2.170.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.587076 9950 raft_consensus.cc:2955] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5a68e1323b54428b9a4b841a924829cb (127.2.170.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.587273 9437 catalog_manager.cc:5654] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 reported cstate change: config changed from index 5 to 6, VOTER 5a68e1323b54428b9a4b841a924829cb (127.2.170.195) evicted. New cstate: current_term: 1 leader_uuid: "f106240450834d39a486b36a4ea76851" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.587491 9421 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 2882613655374b0ab2fedbfb0676349d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:38.587935 9675 tablet_service.cc:1552] Processing DeleteTablet for tablet 2882613655374b0ab2fedbfb0676349d with delete_type TABLET_DATA_TOMBSTONED (TS 5a68e1323b54428b9a4b841a924829cb not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49650
I20251209 03:05:38.588130 10036 tablet_replica.cc:333] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: stopping tablet replica
I20251209 03:05:38.588192 10036 raft_consensus.cc:2243] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:38.588255 10036 raft_consensus.cc:2272] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:38.588507 10036 ts_tablet_manager.cc:1916] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:05:38.589830 10036 ts_tablet_manager.cc:1929] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251209 03:05:38.589887 10036 log.cc:1199] T 2882613655374b0ab2fedbfb0676349d P 5a68e1323b54428b9a4b841a924829cb: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-2-root/wals/2882613655374b0ab2fedbfb0676349d
I20251209 03:05:38.590164 9422 catalog_manager.cc:4985] TS 5a68e1323b54428b9a4b841a924829cb (127.2.170.195:44713): tablet 2882613655374b0ab2fedbfb0676349d (table test-workload [id=c433a9ff555e4c788994a06208f4ccb0]) successfully deleted
I20251209 03:05:38.614272 10023 raft_consensus.cc:1064] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb: attempting to promote NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 to VOTER
I20251209 03:05:38.614521 10023 consensus_queue.cc:237] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } }
I20251209 03:05:38.615023 9871 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.615078 9950 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.615078 9615 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.615278 10023 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.615403 10038 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.615485 9745 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.616240 10023 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.616657 9950 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.617529 9435 catalog_manager.cc:5654] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb reported cstate change: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:05:38.617741 9871 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.618408 9690 consensus_queue.cc:237] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } }
I20251209 03:05:38.618783 9871 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251209 03:05:38.618861 9950 raft_consensus.cc:1275] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251209 03:05:38.619158 10038 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251209 03:05:38.619252 9745 consensus_queue.cc:1048] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251209 03:05:38.619886 10022 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f106240450834d39a486b36a4ea76851 (127.2.170.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.619992 9871 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f106240450834d39a486b36a4ea76851 (127.2.170.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.620868 9950 raft_consensus.cc:2955] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f106240450834d39a486b36a4ea76851 (127.2.170.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.621003 9437 catalog_manager.cc:5654] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb reported cstate change: config changed from index 5 to 6, VOTER f106240450834d39a486b36a4ea76851 (127.2.170.194) evicted. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:05:38.621716 9604 tablet_service.cc:1552] Processing DeleteTablet for tablet 4135fcbfa27d4b438a22b2319f49bbd4 with delete_type TABLET_DATA_TOMBSTONED (TS f106240450834d39a486b36a4ea76851 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34546
I20251209 03:05:38.621759 9422 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet 4135fcbfa27d4b438a22b2319f49bbd4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:38.622066 10042 tablet_replica.cc:333] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: stopping tablet replica
I20251209 03:05:38.622213 10042 raft_consensus.cc:2243] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:38.622282 10042 pending_rounds.cc:70] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: Trying to abort 1 pending ops.
I20251209 03:05:38.622308 10042 pending_rounds.cc:77] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: Aborting op as it isn't in flight: id { term: 1 index: 5 } timestamp: 7230462110164758528 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "4135fcbfa27d4b438a22b2319f49bbd4" old_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: NON_VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: true } } } new_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } attrs { replace: true } } peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } } }
I20251209 03:05:38.622406 10042 raft_consensus.cc:2917] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Aborting config change with OpId 1.5: Aborted: Op aborted
I20251209 03:05:38.622447 10042 raft_consensus.cc:2889] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20251209 03:05:38.622471 10042 raft_consensus.cc:2272] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:38.622773 10042 ts_tablet_manager.cc:1916] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:05:38.624256 10042 ts_tablet_manager.cc:1929] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251209 03:05:38.624321 10042 log.cc:1199] T 4135fcbfa27d4b438a22b2319f49bbd4 P f106240450834d39a486b36a4ea76851: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-1-root/wals/4135fcbfa27d4b438a22b2319f49bbd4
I20251209 03:05:38.624629 9421 catalog_manager.cc:4985] TS f106240450834d39a486b36a4ea76851 (127.2.170.194:38749): tablet 4135fcbfa27d4b438a22b2319f49bbd4 (table test-workload [id=c433a9ff555e4c788994a06208f4ccb0]) successfully deleted
I20251209 03:05:38.733124 10038 raft_consensus.cc:1064] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb: attempting to promote NON_VOTER 60069bea3a4d425d9176c6c96143e2c4 to VOTER
I20251209 03:05:38.733351 10038 consensus_queue.cc:237] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } }
I20251209 03:05:38.733879 9950 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.733881 9615 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.733964 9871 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 LEARNER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20251209 03:05:38.734150 9745 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.734264 10022 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.734404 10023 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251209 03:05:38.735069 9745 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.735215 9871 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.736133 9950 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.736340 9437 catalog_manager.cc:5654] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb reported cstate change: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20251209 03:05:38.737121 9615 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 60069bea3a4d425d9176c6c96143e2c4 (127.2.170.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aa74703951b844639caf72b3bc20d475" member_type: VOTER last_known_addr { host: "127.2.170.193" port: 43401 } attrs { replace: true } } peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.737449 9691 consensus_queue.cc:237] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } }
I20251209 03:05:38.738045 9616 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251209 03:05:38.738363 10023 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251209 03:05:38.738469 9871 raft_consensus.cc:1275] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Refusing update from remote peer 5a68e1323b54428b9a4b841a924829cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20251209 03:05:38.738673 10023 consensus_queue.cc:1048] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251209 03:05:38.738983 10023 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER aa74703951b844639caf72b3bc20d475 (127.2.170.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.739063 9616 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER aa74703951b844639caf72b3bc20d475 (127.2.170.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.739291 9871 raft_consensus.cc:2955] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER aa74703951b844639caf72b3bc20d475 (127.2.170.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.740286 9437 catalog_manager.cc:5654] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 reported cstate change: config changed from index 5 to 6, VOTER aa74703951b844639caf72b3bc20d475 (127.2.170.193) evicted. New cstate: current_term: 1 leader_uuid: "5a68e1323b54428b9a4b841a924829cb" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f106240450834d39a486b36a4ea76851" member_type: VOTER last_known_addr { host: "127.2.170.194" port: 38749 } } peers { permanent_uuid: "5a68e1323b54428b9a4b841a924829cb" member_type: VOTER last_known_addr { host: "127.2.170.195" port: 44713 } } peers { permanent_uuid: "60069bea3a4d425d9176c6c96143e2c4" member_type: VOTER last_known_addr { host: "127.2.170.196" port: 44759 } attrs { promote: false } } }
I20251209 03:05:38.740746 9422 catalog_manager.cc:5167] ChangeConfig:REMOVE_PEER RPC for tablet b2c804e735df449eab77a465a518f6ff with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251209 03:05:38.741008 9940 tablet_service.cc:1552] Processing DeleteTablet for tablet b2c804e735df449eab77a465a518f6ff with delete_type TABLET_DATA_TOMBSTONED (TS aa74703951b844639caf72b3bc20d475 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:40184
I20251209 03:05:38.741225 10047 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:38.741290 10047 raft_consensus.cc:2243] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:38.741357 10047 raft_consensus.cc:2272] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:38.741580 10047 ts_tablet_manager.cc:1916] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251209 03:05:38.742859 10047 ts_tablet_manager.cc:1929] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20251209 03:05:38.742913 10047 log.cc:1199] T b2c804e735df449eab77a465a518f6ff P aa74703951b844639caf72b3bc20d475: Deleting WAL directory at /tmp/dist-test-task3WyU3I/test-tmp/auto_rebalancer-test.0.AutoRebalancerTest.TestDeletedTables.1765249356537226-2731-0/minicluster-data/ts-0-root/wals/b2c804e735df449eab77a465a518f6ff
I20251209 03:05:38.743266 9424 catalog_manager.cc:4985] TS aa74703951b844639caf72b3bc20d475 (127.2.170.193:43401): tablet b2c804e735df449eab77a465a518f6ff (table test-workload [id=c433a9ff555e4c788994a06208f4ccb0]) successfully deleted
W20251209 03:05:39.741061 9486 auto_rebalancer.cc:227] Could not retrieve cluster info: Not found: tserver 5a68e1323b54428b9a4b841a924829cb not available for placement
I20251209 03:05:40.865967 2731 tablet_server.cc:178] TabletServer@127.2.170.193:43401 shutting down...
I20251209 03:05:40.868355 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:40.868518 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:40.868577 2731 raft_consensus.cc:2243] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:40.868657 2731 raft_consensus.cc:2272] T 4135fcbfa27d4b438a22b2319f49bbd4 P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.868839 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:40.868871 2731 raft_consensus.cc:2243] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:40.868930 2731 raft_consensus.cc:2272] T 2882613655374b0ab2fedbfb0676349d P aa74703951b844639caf72b3bc20d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.869040 2731 tablet_replica.cc:333] stopping tablet replica
I20251209 03:05:40.869074 2731 raft_consensus.cc:2243] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:40.869095 2731 raft_consensus.cc:2272] T ed91b26a88e54442888350505b700d79 P aa74703951b844639caf72b3bc20d475 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.880899 2731 tablet_server.cc:195] TabletServer@127.2.170.193:43401 shutdown complete.
I20251209 03:05:40.882407 2731 tablet_server.cc:178] TabletServer@127.2.170.194:0 shutting down...
I20251209 03:05:40.885836 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:40.885963 2731 tablet_replica.cc:333] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851: stopping tablet replica
I20251209 03:05:40.886013 2731 raft_consensus.cc:2243] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:40.886094 2731 raft_consensus.cc:2272] T b2c804e735df449eab77a465a518f6ff P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.886279 2731 tablet_replica.cc:333] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851: stopping tablet replica
I20251209 03:05:40.886315 2731 raft_consensus.cc:2243] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:40.886356 2731 raft_consensus.cc:2272] T ed91b26a88e54442888350505b700d79 P f106240450834d39a486b36a4ea76851 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.886467 2731 tablet_replica.cc:333] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851: stopping tablet replica
I20251209 03:05:40.886494 2731 raft_consensus.cc:2243] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:40.886544 2731 raft_consensus.cc:2272] T 2882613655374b0ab2fedbfb0676349d P f106240450834d39a486b36a4ea76851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.890208 2731 tablet_server.cc:195] TabletServer@127.2.170.194:0 shutdown complete.
I20251209 03:05:40.891760 2731 tablet_server.cc:178] TabletServer@127.2.170.195:0 shutting down...
I20251209 03:05:40.895138 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:40.895254 2731 tablet_replica.cc:333] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb: stopping tablet replica
I20251209 03:05:40.895313 2731 raft_consensus.cc:2243] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:40.895393 2731 raft_consensus.cc:2272] T 4135fcbfa27d4b438a22b2319f49bbd4 P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.895547 2731 tablet_replica.cc:333] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb: stopping tablet replica
I20251209 03:05:40.895586 2731 raft_consensus.cc:2243] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 2 LEADER]: Raft consensus shutting down.
I20251209 03:05:40.895622 2731 raft_consensus.cc:2272] T ed91b26a88e54442888350505b700d79 P 5a68e1323b54428b9a4b841a924829cb [term 2 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.895706 2731 tablet_replica.cc:333] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb: stopping tablet replica
I20251209 03:05:40.895733 2731 raft_consensus.cc:2243] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:40.895767 2731 raft_consensus.cc:2272] T b2c804e735df449eab77a465a518f6ff P 5a68e1323b54428b9a4b841a924829cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.907564 2731 tablet_server.cc:195] TabletServer@127.2.170.195:0 shutdown complete.
I20251209 03:05:40.908938 2731 tablet_server.cc:178] TabletServer@127.2.170.196:0 shutting down...
I20251209 03:05:40.911547 2731 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251209 03:05:40.911672 2731 tablet_replica.cc:333] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4: stopping tablet replica
I20251209 03:05:40.911725 2731 raft_consensus.cc:2243] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:40.911798 2731 raft_consensus.cc:2272] T 2882613655374b0ab2fedbfb0676349d P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.911984 2731 tablet_replica.cc:333] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4: stopping tablet replica
I20251209 03:05:40.912027 2731 raft_consensus.cc:2243] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:40.912076 2731 raft_consensus.cc:2272] T b2c804e735df449eab77a465a518f6ff P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.912190 2731 tablet_replica.cc:333] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4: stopping tablet replica
I20251209 03:05:40.912220 2731 raft_consensus.cc:2243] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251209 03:05:40.912268 2731 raft_consensus.cc:2272] T 4135fcbfa27d4b438a22b2319f49bbd4 P 60069bea3a4d425d9176c6c96143e2c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.923831 2731 tablet_server.cc:195] TabletServer@127.2.170.196:0 shutdown complete.
I20251209 03:05:40.925067 2731 master.cc:561] Master@127.2.170.254:41961 shutting down...
I20251209 03:05:40.927439 2731 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 1 LEADER]: Raft consensus shutting down.
I20251209 03:05:40.927524 2731 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251209 03:05:40.927546 2731 tablet_replica.cc:333] T 00000000000000000000000000000000 P e81c8b1e88c549b18ba480b79eba8d47: stopping tablet replica
I20251209 03:05:40.939370 2731 master.cc:583] Master@127.2.170.254:41961 shutdown complete.
[ OK ] AutoRebalancerTest.TestDeletedTables (7882 ms)
[----------] 14 tests from AutoRebalancerTest (184403 ms total)
[----------] Global test environment tear-down
[==========] 14 tests from 1 test suite ran. (184403 ms total)
[ PASSED ] 13 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AutoRebalancerTest.TestMaxMovesPerServer
1 FAILED TEST
I20251209 03:05:40.945355 2731 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 7 messages since previous log ~4 seconds ago
I20251209 03:05:40.945400 2731 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/ts_tablet_manager.cc:542: suppressed but not reported on 7 messages since previous log ~4 seconds ago
I20251209 03:05:40.945416 2731 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/catalog_manager.cc:4712: suppressed but not reported on 3 messages since previous log ~4 seconds ago
I20251209 03:05:40.945439 2731 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/master/catalog_manager.cc:5332: suppressed but not reported on 12 messages since previous log ~29 seconds ago
I20251209 03:05:40.945466 2731 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 11 messages since previous log ~171 seconds ago
I20251209 03:05:40.945487 2731 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 49 messages since previous log ~17 seconds ago
I20251209 03:05:40.945565 2731 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 21 messages since previous log ~7 seconds ago